Sep 10 22:01:09.743169 ip-10-0-139-247 systemd[1]: Starting Kubernetes Kubelet... Sep 10 22:01:10.225815 ip-10-0-139-247 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 22:01:10.226423 ip-10-0-139-247 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 22:01:10.226423 ip-10-0-139-247 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 22:01:10.226423 ip-10-0-139-247 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 22:01:10.226423 ip-10-0-139-247 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 22:01:10.227720 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.227623 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 22:01:10.231730 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231708 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 22:01:10.231730 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231729 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231733 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231736 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231739 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231742 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231745 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231747 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231750 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231753 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231755 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231760 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231763 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231774 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231780 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231785 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231789 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231792 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231807 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231810 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 22:01:10.231815 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231813 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231816 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231819 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231823 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231827 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231830 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231833 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231835 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231838 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231840 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231843 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231845 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231848 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231851 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231853 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231856 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231858 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231861 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231863 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231866 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 22:01:10.232270 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231868 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231871 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231874 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231876 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231879 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231881 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231884 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231886 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231890 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231895 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231898 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231901 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231903 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231906 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231910 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231913 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231916 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231919 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231921 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231924 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 22:01:10.232759 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231927 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231929 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231932 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231935 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231937 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231940 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231942 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231945 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231947 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231950 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231954 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231957 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231959 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231962 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231964 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231967 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231969 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231972 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231974 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231977 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 22:01:10.233253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231979 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231982 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231984 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231987 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231990 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231992 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231995 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.231998 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232001 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232003 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232006 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232008 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232011 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232013 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232016 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232019 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232021 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232023 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232026 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232744 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 22:01:10.233723 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232752 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232754 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232758 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232761 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232765 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232768 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232771 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232774 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232777 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232781 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232785 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232788 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232791 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232794 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232812 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232817 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232820 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232823 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232826 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 22:01:10.234224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232829 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232831 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232834 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232837 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232840 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232843 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232845 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232848 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232851 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232853 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232856 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232858 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232861 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232863 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232866 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232869 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232871 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232874 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232877 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232880 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 22:01:10.234702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232882 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232885 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232888 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232890 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232894 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232896 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232899 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232901 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232904 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232907 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232911 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232915 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232918 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232920 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232923 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232926 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232929 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232931 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232934 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 22:01:10.235224 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232937 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232939 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232942 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232944 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232947 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232949 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232952 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232954 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232957 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232960 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232962 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232965 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232968 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232971 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232973 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232976 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232979 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232982 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232985 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232987 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 22:01:10.235697 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232990 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232992 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232995 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.232998 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233000 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233003 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233005 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233008 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233011 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233014 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233016 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233019 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233027 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233029 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233032 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233035 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233038 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233040 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233043 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233046 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 22:01:10.236253 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233439 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233444 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233447 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233449 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233453 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233456 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233458 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233461 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233463 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233466 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233468 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233471 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233474 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233478 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233482 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233487 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233490 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233492 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233495 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 22:01:10.236785 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233498 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233500 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233503 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233506 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233508 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233511 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233514 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233517 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233519 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233522 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233524 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233527 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233529 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233532 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233534 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233536 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233539 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233541 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233545 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233547 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 22:01:10.237262 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233550 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233553 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233556 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233558 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233561 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233563 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233566 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233568 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233572 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233575 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233578 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233580 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233582 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233585 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233587 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233590 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233592 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233595 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233597 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233599 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 22:01:10.237746 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233602 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233606 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233609 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233612 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233615 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233618 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233622 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233625 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233627 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233629 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233632 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233635 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233638 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233640 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233643 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233645 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233648 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233650 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233653 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 22:01:10.238272 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233655 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233658 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233660 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233663 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233665 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233668 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233670 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233673 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233675 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233678 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233680 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233683 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233685 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233688 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233690 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233693 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233695 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233698 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233700 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233703 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 22:01:10.238736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.233705 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233786 2576 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233836 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233843 2576 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233847 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233853 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233856 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233860 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233865 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233869 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233872 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233876 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233879 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233882 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233885 2576 flags.go:64] FLAG: --cgroup-root="" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233888 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233891 2576 flags.go:64] FLAG: --client-ca-file="" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233894 2576 flags.go:64] FLAG: --cloud-provider="external" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233897 2576 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233903 2576 flags.go:64] FLAG: --cluster-domain="" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233906 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233909 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233912 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233915 2576 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 22:01:10.239234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233919 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233922 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233925 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233928 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233932 2576 flags.go:64] FLAG: --contention-profiling="false" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233934 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233937 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233941 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233944 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233949 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233952 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233955 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233960 2576 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233963 2576 flags.go:64] FLAG: --enable-server="true" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233966 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233971 2576 flags.go:64] FLAG: --event-burst="100" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233974 2576 flags.go:64] FLAG: --event-qps="50" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233978 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233981 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233984 2576 flags.go:64] FLAG: --eviction-hard="" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233990 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233993 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233996 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.233999 2576 flags.go:64] FLAG: --eviction-soft="" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234001 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 22:01:10.239870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234004 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234007 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234010 2576 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234013 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234016 2576 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234019 2576 flags.go:64] FLAG: --feature-gates="" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234023 2576 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234026 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234029 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234032 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234035 2576 flags.go:64] FLAG: --healthz-port="10248" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234039 2576 flags.go:64] FLAG: --help="false" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234042 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234044 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234047 2576 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234050 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234054 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234057 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234060 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234064 2576 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234067 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234070 2576 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234073 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234077 2576 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 22:01:10.240493 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234082 2576 flags.go:64] FLAG: --kube-reserved="" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234085 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234087 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234091 2576 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234093 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234096 2576 flags.go:64] FLAG: --lock-file="" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234099 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234102 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234105 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234110 2576 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234113 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234116 2576 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234119 2576 flags.go:64] FLAG: --logging-format="text" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234122 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234125 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234128 2576 flags.go:64] FLAG: --manifest-url="" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234131 2576 flags.go:64] FLAG: --manifest-url-header="" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234135 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234138 2576 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234142 2576 flags.go:64] FLAG: --max-pods="110" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234146 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234149 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234151 2576 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234154 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234157 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 22:01:10.241341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234160 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234163 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234172 2576 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234175 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234178 2576 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234181 2576 flags.go:64] FLAG: --pod-cidr="" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234185 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234192 2576 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234196 2576 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234199 2576 flags.go:64] FLAG: --pods-per-core="0" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234202 2576 flags.go:64] FLAG: --port="10250" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234205 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234208 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0386706702610b801" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234212 2576 flags.go:64] FLAG: --qos-reserved="" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234215 2576 flags.go:64] FLAG: --read-only-port="10255" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234218 2576 flags.go:64] FLAG: --register-node="true" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234220 2576 flags.go:64] FLAG: --register-with-taints="" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234224 2576 flags.go:64] FLAG: --registry-burst="10" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234227 2576 flags.go:64] FLAG: --registry-qps="5" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234230 2576 flags.go:64] FLAG: --reserved-cpus="" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234233 2576 flags.go:64] FLAG: --reserved-memory="" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234237 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234240 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234243 2576 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234246 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 22:01:10.242436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234249 2576 flags.go:64] FLAG: --runonce="false" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234252 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234255 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234257 2576 flags.go:64] FLAG: --seccomp-default="false" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234261 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234263 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234266 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234269 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234272 2576 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234277 2576 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234280 2576 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234282 2576 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234285 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234288 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234292 2576 flags.go:64] FLAG: --system-cgroups="" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234296 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234301 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234304 2576 flags.go:64] FLAG: --tls-cert-file="" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234307 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234311 2576 flags.go:64] FLAG: --tls-min-version="" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234314 2576 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234317 2576 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234319 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234322 2576 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234325 2576 flags.go:64] FLAG: --v="2" Sep 10 22:01:10.243414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234329 2576 flags.go:64] FLAG: --version="false" Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234334 2576 flags.go:64] FLAG: --vmodule="" Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234337 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.234340 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234478 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234482 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234486 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234488 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234491 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234494 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234496 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234499 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234502 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234504 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234507 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234509 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234514 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234516 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234519 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234522 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 22:01:10.244192 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234524 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234527 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234531 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234534 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234537 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234540 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234542 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234545 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234549 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234553 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234556 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234559 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234562 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234564 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234567 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234570 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234572 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234576 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234578 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 22:01:10.244700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234582 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234585 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234588 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234592 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234596 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234599 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234602 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234605 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234607 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234611 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234614 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234616 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234619 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234622 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234624 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234628 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234632 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234634 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234637 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 22:01:10.245188 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234639 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234642 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234644 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234647 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234650 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234652 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234655 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234657 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234660 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234663 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234665 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234668 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234670 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234673 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234676 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234678 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234681 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234683 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234686 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 22:01:10.245760 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234688 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234691 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234694 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234697 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234700 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234703 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234705 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234708 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234711 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234715 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234717 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234721 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234723 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234726 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234729 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234731 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234734 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234737 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234739 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234742 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 22:01:10.246612 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234745 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 22:01:10.247345 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234747 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 22:01:10.247345 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234750 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 22:01:10.247345 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234753 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 22:01:10.247345 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234755 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 22:01:10.247345 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.234758 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 22:01:10.247345 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.235322 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 22:01:10.247345 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.243062 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 22:01:10.247345 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.243098 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 22:01:10.247345 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.245534 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 22:01:10.247345 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.245561 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 22:01:10.247345 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245668 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 22:01:10.247345 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245677 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 22:01:10.247345 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245683 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 22:01:10.247345 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245687 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 22:01:10.247345 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245692 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245696 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245700 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245705 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245709 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245713 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245718 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245722 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245727 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245732 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245736 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245740 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245747 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245752 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245757 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245762 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245766 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245770 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245774 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245779 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 22:01:10.247963 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245783 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245787 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245794 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245815 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245820 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245827 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245832 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245837 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245842 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245848 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245852 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245856 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245860 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245865 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245870 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245874 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245878 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245883 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245887 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245892 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 22:01:10.248790 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245896 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245900 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245905 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245909 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245913 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245918 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245922 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245926 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245930 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245934 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245938 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245942 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245947 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245952 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245956 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245961 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245965 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245970 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245977 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 22:01:10.249490 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245981 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245989 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.245995 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246000 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246004 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246010 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246015 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246020 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246024 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246028 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246032 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246037 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246041 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246045 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246049 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246054 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246058 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246062 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246067 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 22:01:10.250163 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246070 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246075 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246080 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246085 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246089 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246093 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246097 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246102 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246106 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246110 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246114 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246120 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246125 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246130 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246135 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246139 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 22:01:10.250667 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246143 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 22:01:10.251125 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.246151 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 22:01:10.251125 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246343 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 22:01:10.251125 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246351 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 22:01:10.251125 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246357 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 22:01:10.251125 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246362 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 22:01:10.251125 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246367 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 22:01:10.251125 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246371 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 22:01:10.251125 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246376 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 22:01:10.251125 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246381 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 22:01:10.251125 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246385 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 22:01:10.251125 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246392 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 22:01:10.251125 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246399 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 22:01:10.251125 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246404 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 22:01:10.251125 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246409 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246413 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246418 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246423 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246427 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246432 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246436 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246440 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246444 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246449 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246453 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246457 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246461 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246466 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246471 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246475 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246481 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246485 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246489 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 22:01:10.251473 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246494 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246498 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246503 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246507 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246511 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246515 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246519 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246523 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246527 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246532 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246536 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246540 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246545 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246549 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246553 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246557 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246561 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246566 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246570 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 22:01:10.251951 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246575 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246581 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246585 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246589 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246593 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246597 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246602 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246607 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246611 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246615 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246619 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246624 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246628 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246634 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246640 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246645 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246650 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246654 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246658 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246662 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 22:01:10.252481 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246667 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246671 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246675 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246679 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246684 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246688 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246692 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246697 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246701 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246706 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246710 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246714 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246719 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246723 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246727 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246731 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246736 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246740 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246744 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246748 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246752 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 22:01:10.252997 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246757 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 22:01:10.253712 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246761 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 22:01:10.253712 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246765 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 22:01:10.253712 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246770 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 22:01:10.253712 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246775 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 22:01:10.253712 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246778 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 22:01:10.253712 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246782 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 22:01:10.253712 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:10.246787 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 10 22:01:10.253712 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.246795 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 22:01:10.253712 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.247000 2576 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 22:01:10.253712 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.249653 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 22:01:10.253712 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.250630 2576 server.go:1019] "Starting client certificate rotation" Sep 10 22:01:10.253712 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.250741 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 22:01:10.253712 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.250828 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 22:01:10.254180 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.253933 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 22:01:10.264828 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.264789 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 22:01:10.277021 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.276995 2576 log.go:25] "Validated CRI v1 runtime API" Sep 10 22:01:10.280397 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.280377 2576 log.go:25] "Validated CRI v1 image API" Sep 10 22:01:10.281327 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.281302 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 22:01:10.285847 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.285824 2576 fs.go:135] Filesystem UUIDs: map[04afa6d2-efdf-4de5-9f33-ddc87d8784d7:/dev/nvme0n1p4 704d77e3-b5e3-43eb-a1f2-43cd94841ee2:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Sep 10 22:01:10.285913 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.285846 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 22:01:10.290246 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.290090 2576 manager.go:216] Machine: {Timestamp:2026-09-10 22:01:10.289582535 +0000 UTC m=+0.411281316 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101240 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec257fd37fdafbfa16d7b40aa8bc0e97 SystemUUID:ec257fd3-7fda-fbfa-16d7-b40aa8bc0e97 BootID:585b0f72-ea86-40a5-b731-be1047ca9bbb Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4a:eb:62:e9:f1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4a:eb:62:e9:f1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:69:61:82:6a:d7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 22:01:10.290246 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.290238 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 22:01:10.290370 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.290329 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 22:01:10.292224 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.292204 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 22:01:10.298344 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.298313 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 22:01:10.298500 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.298347 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-247.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 22:01:10.300514 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.300501 2576 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 22:01:10.300556 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.300517 2576 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 22:01:10.300556 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.300531 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 22:01:10.301281 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.301269 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 22:01:10.302045 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.302032 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 22:01:10.306461 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.306445 2576 state_mem.go:36] "Initialized new in-memory state store" Sep 10 22:01:10.306586 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.306577 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 22:01:10.306620 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.306608 2576 kubelet.go:486] "Attempting to sync node with API server" Sep 10 22:01:10.306620 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.306618 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 22:01:10.306679 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.306634 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 22:01:10.306679 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.306644 2576 kubelet.go:398] "Adding apiserver pod source" Sep 10 22:01:10.306679 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.306654 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 22:01:10.307086 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.307073 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 22:01:10.307127 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.307094 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 22:01:10.313413 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.313389 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 22:01:10.315744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.315725 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 22:01:10.315881 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.315757 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 22:01:10.318478 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.318463 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 22:01:10.318535 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.318482 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 22:01:10.318535 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.318488 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 22:01:10.318535 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.318501 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 22:01:10.318535 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.318510 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 22:01:10.318535 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.318517 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 22:01:10.318535 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.318530 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 22:01:10.318535 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.318536 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 22:01:10.318758 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.318543 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 22:01:10.318758 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.318549 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 22:01:10.318758 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.318563 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 22:01:10.318758 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.318569 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 22:01:10.319268 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.319258 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 22:01:10.319268 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.319267 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 22:01:10.323141 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.323103 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-247.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 22:01:10.323141 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.323133 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 22:01:10.324743 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.324722 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-247.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 22:01:10.325266 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.325253 2576 server.go:1268] "Started kubelet" Sep 10 22:01:10.325339 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.325309 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 22:01:10.325400 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.325346 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 22:01:10.325434 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.325426 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 22:01:10.326182 ip-10-0-139-247 systemd[1]: Started Kubernetes Kubelet. Sep 10 22:01:10.326685 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.326664 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 22:01:10.327912 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.327898 2576 server.go:310] "Adding debug handlers to kubelet server" Sep 10 22:01:10.329568 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.329548 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 22:01:10.332055 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.332037 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 22:01:10.332144 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.332100 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 22:01:10.332200 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.332170 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-247.ec2.internal\" not found" Sep 10 22:01:10.332900 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.332863 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 22:01:10.332900 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.332862 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 22:01:10.332900 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.332895 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 22:01:10.333091 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.332959 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 22:01:10.333091 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.332975 2576 factory.go:55] Registering systemd factory Sep 10 22:01:10.333091 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.332985 2576 factory.go:223] Registration of the systemd container factory successfully Sep 10 22:01:10.333091 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.332993 2576 reconstruct.go:97] "Volume reconstruction finished" Sep 10 22:01:10.333091 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.333003 2576 reconciler.go:29] "Reconciler: start to sync state" Sep 10 22:01:10.333334 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.333317 2576 factory.go:153] Registering CRI-O factory Sep 10 22:01:10.333334 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.333331 2576 factory.go:223] Registration of the crio container factory successfully Sep 10 22:01:10.333422 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.333366 2576 factory.go:103] Registering Raw factory Sep 10 22:01:10.333422 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.333382 2576 manager.go:1196] Started watching for new ooms in manager Sep 10 22:01:10.333767 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.333755 2576 manager.go:349] Starting recovery of all containers Sep 10 22:01:10.336291 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.336267 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 22:01:10.336850 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.336828 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-247.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 22:01:10.338333 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.336379 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-247.ec2.internal.18d414a987514b9e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-247.ec2.internal,UID:ip-10-0-139-247.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-247.ec2.internal,},FirstTimestamp:2026-09-10 22:01:10.325210014 +0000 UTC m=+0.446908787,LastTimestamp:2026-09-10 22:01:10.325210014 +0000 UTC m=+0.446908787,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-247.ec2.internal,}" Sep 10 22:01:10.344131 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.344110 2576 manager.go:354] Recovery completed Sep 10 22:01:10.350569 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.350553 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 22:01:10.353593 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.353562 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientMemory" Sep 10 22:01:10.353722 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.353602 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 22:01:10.353722 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.353614 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientPID" Sep 10 22:01:10.354258 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.354238 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 22:01:10.354258 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.354257 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 22:01:10.354371 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.354266 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 22:01:10.354371 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.354276 2576 state_mem.go:36] "Initialized new in-memory state store" Sep 10 22:01:10.355102 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.355084 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientMemory" Sep 10 22:01:10.355102 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.355114 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 22:01:10.355102 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.355125 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientPID" Sep 10 22:01:10.355594 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.355582 2576 policy_none.go:49] "None policy: Start" Sep 10 22:01:10.355624 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.355599 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 22:01:10.355624 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.355609 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 22:01:10.356034 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.355945 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-247.ec2.internal.18d414a989024ed2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-247.ec2.internal,UID:ip-10-0-139-247.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-247.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-247.ec2.internal,},FirstTimestamp:2026-09-10 22:01:10.353587922 +0000 UTC m=+0.475286701,LastTimestamp:2026-09-10 22:01:10.353587922 +0000 UTC m=+0.475286701,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-247.ec2.internal,}" Sep 10 22:01:10.356641 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.356621 2576 policy_none.go:47] "Start" Sep 10 22:01:10.365929 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.365852 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-247.ec2.internal.18d414a989029ce2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-247.ec2.internal,UID:ip-10-0-139-247.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-247.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-247.ec2.internal,},FirstTimestamp:2026-09-10 22:01:10.353607906 +0000 UTC m=+0.475306679,LastTimestamp:2026-09-10 22:01:10.353607906 +0000 UTC m=+0.475306679,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-247.ec2.internal,}" Sep 10 22:01:10.367624 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.367557 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-247.ec2.internal.18d414a98902c1f2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-247.ec2.internal,UID:ip-10-0-139-247.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-247.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-247.ec2.internal,},FirstTimestamp:2026-09-10 22:01:10.353617394 +0000 UTC m=+0.475316166,LastTimestamp:2026-09-10 22:01:10.353617394 +0000 UTC m=+0.475316166,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-247.ec2.internal,}" Sep 10 22:01:10.376321 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.376296 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c2chl" Sep 10 22:01:10.380580 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.380508 2576 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-139-247.ec2.internal.18d414a989024ed2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-247.ec2.internal.18d414a989024ed2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-247.ec2.internal,UID:ip-10-0-139-247.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-247.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-247.ec2.internal,},FirstTimestamp:2026-09-10 22:01:10.353587922 +0000 UTC m=+0.475286701,LastTimestamp:2026-09-10 22:01:10.355101063 +0000 UTC m=+0.476799836,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-247.ec2.internal,}" Sep 10 22:01:10.397066 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.383252 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c2chl" Sep 10 22:01:10.397066 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.395659 2576 manager.go:337] "Starting Device Plugin manager" Sep 10 22:01:10.397066 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.395695 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 22:01:10.397066 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.395705 2576 server.go:87] "Starting device plugin registration server" Sep 10 22:01:10.397066 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.396081 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 22:01:10.397066 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.396093 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 22:01:10.397066 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.396183 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 22:01:10.397066 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.396258 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 22:01:10.397066 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.396266 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 22:01:10.397066 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.396853 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 22:01:10.397066 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.396909 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-247.ec2.internal\" not found" Sep 10 22:01:10.423690 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.423647 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 22:01:10.424852 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.424835 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 22:01:10.424959 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.424896 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 22:01:10.424959 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.424929 2576 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 22:01:10.425051 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.424975 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 22:01:10.428673 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.428652 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 22:01:10.496518 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.496402 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 22:01:10.497460 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.497438 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientMemory" Sep 10 22:01:10.497593 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.497476 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 22:01:10.497593 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.497490 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientPID" Sep 10 22:01:10.497593 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.497519 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.507708 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.507684 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.507864 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.507716 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-247.ec2.internal\": node \"ip-10-0-139-247.ec2.internal\" not found" Sep 10 22:01:10.525054 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.525026 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-247.ec2.internal"] Sep 10 22:01:10.525161 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.525103 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 22:01:10.527617 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.527601 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientMemory" Sep 10 22:01:10.527704 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.527630 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 22:01:10.527704 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.527640 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientPID" Sep 10 22:01:10.528999 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.528985 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 22:01:10.529144 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.529127 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.529200 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.529163 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 22:01:10.529947 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.529929 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientMemory" Sep 10 22:01:10.530019 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.529962 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 22:01:10.530019 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.529973 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientPID" Sep 10 22:01:10.530093 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.529929 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientMemory" Sep 10 22:01:10.530093 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.530036 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 22:01:10.530093 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.530053 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientPID" Sep 10 22:01:10.531836 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.531820 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.531914 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.531847 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 22:01:10.532589 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.532574 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientMemory" Sep 10 22:01:10.532650 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.532599 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 22:01:10.532650 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.532609 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientPID" Sep 10 22:01:10.533289 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.533273 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5f12632b0c737aeacd907582d37aedae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal\" (UID: \"5f12632b0c737aeacd907582d37aedae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.533394 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.533302 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f12632b0c737aeacd907582d37aedae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal\" (UID: \"5f12632b0c737aeacd907582d37aedae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.533394 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.533328 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6969b8244f652594d09b7861b9d8eb44-config\") pod \"kube-apiserver-proxy-ip-10-0-139-247.ec2.internal\" (UID: \"6969b8244f652594d09b7861b9d8eb44\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.544254 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.544227 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-247.ec2.internal\" not found" Sep 10 22:01:10.545495 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.545476 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-247.ec2.internal\" not found" node="ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.549007 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.548981 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-247.ec2.internal\" not found" node="ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.633878 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.633841 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5f12632b0c737aeacd907582d37aedae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal\" (UID: \"5f12632b0c737aeacd907582d37aedae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.634059 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.633910 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5f12632b0c737aeacd907582d37aedae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal\" (UID: \"5f12632b0c737aeacd907582d37aedae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.634059 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.633900 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f12632b0c737aeacd907582d37aedae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal\" (UID: \"5f12632b0c737aeacd907582d37aedae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.634059 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.633963 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f12632b0c737aeacd907582d37aedae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal\" (UID: \"5f12632b0c737aeacd907582d37aedae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.634059 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.633980 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6969b8244f652594d09b7861b9d8eb44-config\") pod \"kube-apiserver-proxy-ip-10-0-139-247.ec2.internal\" (UID: \"6969b8244f652594d09b7861b9d8eb44\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.634059 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.634010 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6969b8244f652594d09b7861b9d8eb44-config\") pod \"kube-apiserver-proxy-ip-10-0-139-247.ec2.internal\" (UID: \"6969b8244f652594d09b7861b9d8eb44\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.644954 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.644922 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-247.ec2.internal\" not found" Sep 10 22:01:10.745310 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.745270 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-247.ec2.internal\" not found" Sep 10 22:01:10.846172 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.846144 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-247.ec2.internal\" not found" Sep 10 22:01:10.848325 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.848295 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.853082 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:10.852371 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-247.ec2.internal" Sep 10 22:01:10.946774 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:10.946730 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-247.ec2.internal\" not found" Sep 10 22:01:11.047280 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:11.047243 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-247.ec2.internal\" not found" Sep 10 22:01:11.147956 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:11.147863 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-247.ec2.internal\" not found" Sep 10 22:01:11.248540 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:11.248506 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-247.ec2.internal\" not found" Sep 10 22:01:11.250689 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.250670 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 22:01:11.250879 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.250860 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 22:01:11.348966 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:11.348934 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-247.ec2.internal\" not found" Sep 10 22:01:11.365431 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.365406 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 22:01:11.366418 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.366398 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientMemory" Sep 10 22:01:11.366540 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.366431 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 22:01:11.366540 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.366445 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeHasSufficientPID" Sep 10 22:01:11.386999 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.386956 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 21:56:10 +0000 UTC" deadline="2028-06-12 11:50:26.506100778 +0000 UTC" Sep 10 22:01:11.386999 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.386991 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15373h49m15.119114428s" Sep 10 22:01:11.444879 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.444787 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 22:01:11.448070 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.448050 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 22:01:11.449708 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:11.449677 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-247.ec2.internal\" not found" Sep 10 22:01:11.532218 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.532187 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 22:01:11.532437 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.532279 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal" Sep 10 22:01:11.550687 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.550662 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 22:01:11.555081 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.555055 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-247.ec2.internal" Sep 10 22:01:11.568442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.568420 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 22:01:11.584520 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:11.584489 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6969b8244f652594d09b7861b9d8eb44.slice/crio-5bdd2beef5675e9b4d7e1cc08f8b46f52c914e98975fe544ae3c2f68e9782113 WatchSource:0}: Error finding container 5bdd2beef5675e9b4d7e1cc08f8b46f52c914e98975fe544ae3c2f68e9782113: Status 404 returned error can't find the container with id 5bdd2beef5675e9b4d7e1cc08f8b46f52c914e98975fe544ae3c2f68e9782113 Sep 10 22:01:11.584820 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:11.584784 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f12632b0c737aeacd907582d37aedae.slice/crio-7aad066e62269c87628c73834c5a8e119d8c7525d9dd55c93f32bc3dc7ec8e72 WatchSource:0}: Error finding container 7aad066e62269c87628c73834c5a8e119d8c7525d9dd55c93f32bc3dc7ec8e72: Status 404 returned error can't find the container with id 7aad066e62269c87628c73834c5a8e119d8c7525d9dd55c93f32bc3dc7ec8e72 Sep 10 22:01:11.589406 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:11.589342 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 22:01:12.307084 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.307038 2576 apiserver.go:52] "Watching apiserver" Sep 10 22:01:12.312910 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.312880 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 22:01:12.314141 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.314109 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-247.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx","openshift-cluster-node-tuning-operator/tuned-mbz8w","openshift-image-registry/node-ca-zfmq5","openshift-multus/multus-additional-cni-plugins-pjbjk","openshift-multus/multus-bj2ck","openshift-multus/network-metrics-daemon-lvd5t","openshift-network-diagnostics/network-check-target-kfbmj","kube-system/global-pull-secret-syncer-6dx6d","kube-system/konnectivity-agent-2vmgg","openshift-dns/node-resolver-8z8zh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal","openshift-network-operator/iptables-alerter-t7h7x","openshift-ovn-kubernetes/ovnkube-node-sk86l"] Sep 10 22:01:12.315870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.315842 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2vmgg" Sep 10 22:01:12.317169 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.317141 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t7h7x" Sep 10 22:01:12.317933 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.317910 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 22:01:12.318067 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.317936 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9nd2t\"" Sep 10 22:01:12.318067 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.317939 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 22:01:12.318618 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.318596 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.319543 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.319519 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 22:01:12.319646 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.319568 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 22:01:12.319646 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.319522 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nngt6\"" Sep 10 22:01:12.319770 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.319579 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 22:01:12.321358 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.320098 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.321358 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.320906 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sc9dj\"" Sep 10 22:01:12.321358 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.321010 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 22:01:12.321358 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.321128 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 22:01:12.322668 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.322475 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.322668 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.322628 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 22:01:12.323597 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.323300 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 22:01:12.323597 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.323387 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c7ksq\"" Sep 10 22:01:12.323597 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.323473 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 22:01:12.323597 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.323478 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 22:01:12.323900 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.323760 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 22:01:12.323961 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.323917 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 22:01:12.324025 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.324007 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.325745 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.325502 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 22:01:12.325851 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.325790 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:12.325920 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:12.325876 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:12.325920 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.325889 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 22:01:12.326838 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.326302 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 22:01:12.326838 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.326379 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 22:01:12.326838 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.326512 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 22:01:12.326838 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.326601 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jg8vm\"" Sep 10 22:01:12.326838 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.326602 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-k6l75\"" Sep 10 22:01:12.326838 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.326700 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 22:01:12.327150 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.327026 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:12.327150 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:12.327086 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:12.328508 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.328489 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8z8zh" Sep 10 22:01:12.330047 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.330030 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.330419 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.330399 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 22:01:12.330419 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.330416 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9dgfc\"" Sep 10 22:01:12.330552 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.330403 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 22:01:12.331330 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.331313 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:12.331417 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:12.331375 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:12.332271 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.332253 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 22:01:12.332485 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.332464 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 22:01:12.332908 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.332860 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qw9d6\"" Sep 10 22:01:12.333786 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.333576 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 22:01:12.333984 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.333962 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zfmq5" Sep 10 22:01:12.334100 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.334081 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 22:01:12.336004 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.335985 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 22:01:12.336095 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.336013 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 22:01:12.336180 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.336164 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 22:01:12.336234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.336181 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 22:01:12.336364 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.336338 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-69g6j\"" Sep 10 22:01:12.342725 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.342702 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-kubernetes\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.342876 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.342728 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-run-ovn\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.342876 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.342782 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/94294796-56b0-4302-8038-441734369d32-ovnkube-config\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.342876 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.342835 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtw57\" (UniqueName: \"kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57\") pod \"network-check-target-kfbmj\" (UID: \"57428f1e-0c76-4457-937b-b33b503cb09a\") " pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:12.342876 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.342862 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/729bf7d2-7285-450d-8a22-f13dd26be9d8-system-cni-dir\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.343080 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.342888 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/729bf7d2-7285-450d-8a22-f13dd26be9d8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.343080 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.342913 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-hostroot\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.343080 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.342936 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-tuned\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.343080 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.342960 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.343080 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343018 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-var-lib-cni-bin\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.343080 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343064 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-lib-modules\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.343322 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343088 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55cp2\" (UniqueName: \"kubernetes.io/projected/94294796-56b0-4302-8038-441734369d32-kube-api-access-55cp2\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.343322 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343125 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a95cad0a-1c31-4ecf-915e-05b752bdaeee-host\") pod \"node-ca-zfmq5\" (UID: \"a95cad0a-1c31-4ecf-915e-05b752bdaeee\") " pod="openshift-image-registry/node-ca-zfmq5" Sep 10 22:01:12.343322 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343163 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-sys-fs\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.343322 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343192 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzfpj\" (UniqueName: \"kubernetes.io/projected/cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6-kube-api-access-pzfpj\") pod \"iptables-alerter-t7h7x\" (UID: \"cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6\") " pod="openshift-network-operator/iptables-alerter-t7h7x" Sep 10 22:01:12.343322 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343229 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjgkb\" (UniqueName: \"kubernetes.io/projected/39f4bef5-867c-4a17-abfe-489b216ffd8c-kube-api-access-sjgkb\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.343322 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343260 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-systemd-units\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.343322 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343294 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx5x5\" (UniqueName: \"kubernetes.io/projected/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-kube-api-access-fx5x5\") pod \"network-metrics-daemon-lvd5t\" (UID: \"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0\") " pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:12.343322 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343318 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52hkf\" (UniqueName: \"kubernetes.io/projected/bd1bec7b-f106-4ca2-b72d-71a7fe7a1929-kube-api-access-52hkf\") pod \"node-resolver-8z8zh\" (UID: \"bd1bec7b-f106-4ca2-b72d-71a7fe7a1929\") " pod="openshift-dns/node-resolver-8z8zh" Sep 10 22:01:12.343657 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343344 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-kubelet-config\") pod \"global-pull-secret-syncer-6dx6d\" (UID: \"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7\") " pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:12.343657 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343367 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-system-cni-dir\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.343657 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343397 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-cnibin\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.343657 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343430 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-run-multus-certs\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.343657 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343460 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-registration-dir\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.343657 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343485 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gmc6\" (UniqueName: \"kubernetes.io/projected/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-kube-api-access-6gmc6\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.343657 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343508 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-sysctl-conf\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.343657 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343530 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-host\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.343657 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343568 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-run-k8s-cni-cncf-io\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.343657 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343607 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-var-lib-cni-multus\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.343657 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343643 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-etc-kubernetes\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343681 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-sysconfig\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343709 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-slash\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343737 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/94294796-56b0-4302-8038-441734369d32-env-overrides\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343764 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/94294796-56b0-4302-8038-441734369d32-ovn-node-metrics-cert\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343787 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs\") pod \"network-metrics-daemon-lvd5t\" (UID: \"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0\") " pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343826 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-multus-socket-dir-parent\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343850 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-run\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343870 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-var-lib-kubelet\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343893 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd1bec7b-f106-4ca2-b72d-71a7fe7a1929-hosts-file\") pod \"node-resolver-8z8zh\" (UID: \"bd1bec7b-f106-4ca2-b72d-71a7fe7a1929\") " pod="openshift-dns/node-resolver-8z8zh" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343915 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd1bec7b-f106-4ca2-b72d-71a7fe7a1929-tmp-dir\") pod \"node-resolver-8z8zh\" (UID: \"bd1bec7b-f106-4ca2-b72d-71a7fe7a1929\") " pod="openshift-dns/node-resolver-8z8zh" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343936 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a95cad0a-1c31-4ecf-915e-05b752bdaeee-serviceca\") pod \"node-ca-zfmq5\" (UID: \"a95cad0a-1c31-4ecf-915e-05b752bdaeee\") " pod="openshift-image-registry/node-ca-zfmq5" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343960 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/729bf7d2-7285-450d-8a22-f13dd26be9d8-cni-binary-copy\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.343990 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/729bf7d2-7285-450d-8a22-f13dd26be9d8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344043 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-var-lib-kubelet\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344077 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-socket-dir\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.344111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344100 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39f4bef5-867c-4a17-abfe-489b216ffd8c-tmp\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344122 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-run-netns\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344154 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a3ae936-7af2-40dc-ac1b-88ed2341dbfe-agent-certs\") pod \"konnectivity-agent-2vmgg\" (UID: \"2a3ae936-7af2-40dc-ac1b-88ed2341dbfe\") " pod="kube-system/konnectivity-agent-2vmgg" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344194 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/729bf7d2-7285-450d-8a22-f13dd26be9d8-cnibin\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344227 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-dbus\") pod \"global-pull-secret-syncer-6dx6d\" (UID: \"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7\") " pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344262 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-etc-selinux\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344286 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-log-socket\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344317 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-cni-bin\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344365 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-cni-netd\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344395 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc6d7\" (UniqueName: \"kubernetes.io/projected/729bf7d2-7285-450d-8a22-f13dd26be9d8-kube-api-access-tc6d7\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344424 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-multus-cni-dir\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344446 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-sys\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344504 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-kubelet\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344542 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-node-log\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344565 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/729bf7d2-7285-450d-8a22-f13dd26be9d8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344600 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-run-netns\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.344921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344625 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnn7w\" (UniqueName: \"kubernetes.io/projected/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-kube-api-access-hnn7w\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344651 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-modprobe-d\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344692 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-sysctl-d\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344725 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-systemd\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344752 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-cni-binary-copy\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344776 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-multus-conf-dir\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344818 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344854 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6-host-slash\") pod \"iptables-alerter-t7h7x\" (UID: \"cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6\") " pod="openshift-network-operator/iptables-alerter-t7h7x" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344869 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344897 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-var-lib-openvswitch\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344937 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-multus-daemon-config\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344965 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret\") pod \"global-pull-secret-syncer-6dx6d\" (UID: \"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7\") " pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.344989 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a3ae936-7af2-40dc-ac1b-88ed2341dbfe-konnectivity-ca\") pod \"konnectivity-agent-2vmgg\" (UID: \"2a3ae936-7af2-40dc-ac1b-88ed2341dbfe\") " pod="kube-system/konnectivity-agent-2vmgg" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.345013 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-run-systemd\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.345034 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-etc-openvswitch\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.345067 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-run-ovn-kubernetes\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.345091 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-run-openvswitch\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.345727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.345116 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/94294796-56b0-4302-8038-441734369d32-ovnkube-script-lib\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.346523 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.345137 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7r4v\" (UniqueName: \"kubernetes.io/projected/a95cad0a-1c31-4ecf-915e-05b752bdaeee-kube-api-access-w7r4v\") pod \"node-ca-zfmq5\" (UID: \"a95cad0a-1c31-4ecf-915e-05b752bdaeee\") " pod="openshift-image-registry/node-ca-zfmq5" Sep 10 22:01:12.346523 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.345150 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-device-dir\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.346523 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.345166 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6-iptables-alerter-script\") pod \"iptables-alerter-t7h7x\" (UID: \"cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6\") " pod="openshift-network-operator/iptables-alerter-t7h7x" Sep 10 22:01:12.378513 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.378480 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pn546" Sep 10 22:01:12.387876 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.387849 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pn546" Sep 10 22:01:12.429658 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.429609 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-247.ec2.internal" event={"ID":"6969b8244f652594d09b7861b9d8eb44","Type":"ContainerStarted","Data":"5bdd2beef5675e9b4d7e1cc08f8b46f52c914e98975fe544ae3c2f68e9782113"} Sep 10 22:01:12.430853 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.430823 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal" event={"ID":"5f12632b0c737aeacd907582d37aedae","Type":"ContainerStarted","Data":"7aad066e62269c87628c73834c5a8e119d8c7525d9dd55c93f32bc3dc7ec8e72"} Sep 10 22:01:12.446007 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.445978 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-var-lib-cni-bin\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.446007 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446012 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-lib-modules\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.446246 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446030 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55cp2\" (UniqueName: \"kubernetes.io/projected/94294796-56b0-4302-8038-441734369d32-kube-api-access-55cp2\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.446246 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446097 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-var-lib-cni-bin\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.446246 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446147 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a95cad0a-1c31-4ecf-915e-05b752bdaeee-host\") pod \"node-ca-zfmq5\" (UID: \"a95cad0a-1c31-4ecf-915e-05b752bdaeee\") " pod="openshift-image-registry/node-ca-zfmq5" Sep 10 22:01:12.446246 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446169 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-sys-fs\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.446246 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446189 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzfpj\" (UniqueName: \"kubernetes.io/projected/cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6-kube-api-access-pzfpj\") pod \"iptables-alerter-t7h7x\" (UID: \"cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6\") " pod="openshift-network-operator/iptables-alerter-t7h7x" Sep 10 22:01:12.446246 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446207 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjgkb\" (UniqueName: \"kubernetes.io/projected/39f4bef5-867c-4a17-abfe-489b216ffd8c-kube-api-access-sjgkb\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.446246 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446230 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-systemd-units\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.446246 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446245 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx5x5\" (UniqueName: \"kubernetes.io/projected/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-kube-api-access-fx5x5\") pod \"network-metrics-daemon-lvd5t\" (UID: \"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0\") " pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:12.446506 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446257 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-lib-modules\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.446506 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446262 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52hkf\" (UniqueName: \"kubernetes.io/projected/bd1bec7b-f106-4ca2-b72d-71a7fe7a1929-kube-api-access-52hkf\") pod \"node-resolver-8z8zh\" (UID: \"bd1bec7b-f106-4ca2-b72d-71a7fe7a1929\") " pod="openshift-dns/node-resolver-8z8zh" Sep 10 22:01:12.446506 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446346 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a95cad0a-1c31-4ecf-915e-05b752bdaeee-host\") pod \"node-ca-zfmq5\" (UID: \"a95cad0a-1c31-4ecf-915e-05b752bdaeee\") " pod="openshift-image-registry/node-ca-zfmq5" Sep 10 22:01:12.446506 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446355 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-kubelet-config\") pod \"global-pull-secret-syncer-6dx6d\" (UID: \"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7\") " pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:12.446506 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446408 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-kubelet-config\") pod \"global-pull-secret-syncer-6dx6d\" (UID: \"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7\") " pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:12.446506 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446314 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-sys-fs\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.446506 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446459 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-systemd-units\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.446709 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446662 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-system-cni-dir\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.446742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446715 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-cnibin\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.446783 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446740 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-run-multus-certs\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.446783 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446761 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-registration-dir\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.446883 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446781 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gmc6\" (UniqueName: \"kubernetes.io/projected/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-kube-api-access-6gmc6\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.446883 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446819 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-sysctl-conf\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.446883 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446842 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-host\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.446883 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446847 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-registration-dir\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.446883 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446850 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-system-cni-dir\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.446883 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446781 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-cnibin\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.446883 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446864 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-run-k8s-cni-cncf-io\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446897 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-run-multus-certs\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446898 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-var-lib-cni-multus\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446930 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-var-lib-cni-multus\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446933 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-host\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446904 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-run-k8s-cni-cncf-io\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446965 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-sysctl-conf\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446961 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-etc-kubernetes\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446995 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-sysconfig\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.446998 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-etc-kubernetes\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447018 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-slash\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447059 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/94294796-56b0-4302-8038-441734369d32-env-overrides\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447065 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-sysconfig\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447084 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/94294796-56b0-4302-8038-441734369d32-ovn-node-metrics-cert\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447110 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs\") pod \"network-metrics-daemon-lvd5t\" (UID: \"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0\") " pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447116 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-slash\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447134 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-multus-socket-dir-parent\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447158 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-run\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.447239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447187 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-var-lib-kubelet\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:12.447204 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447210 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd1bec7b-f106-4ca2-b72d-71a7fe7a1929-hosts-file\") pod \"node-resolver-8z8zh\" (UID: \"bd1bec7b-f106-4ca2-b72d-71a7fe7a1929\") " pod="openshift-dns/node-resolver-8z8zh" Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447218 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-multus-socket-dir-parent\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447225 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-run\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447232 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd1bec7b-f106-4ca2-b72d-71a7fe7a1929-tmp-dir\") pod \"node-resolver-8z8zh\" (UID: \"bd1bec7b-f106-4ca2-b72d-71a7fe7a1929\") " pod="openshift-dns/node-resolver-8z8zh" Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:12.447269 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs podName:6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0 nodeName:}" failed. No retries permitted until 2026-09-10 22:01:12.947248206 +0000 UTC m=+3.068946979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs") pod "network-metrics-daemon-lvd5t" (UID: "6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447272 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-var-lib-kubelet\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447289 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a95cad0a-1c31-4ecf-915e-05b752bdaeee-serviceca\") pod \"node-ca-zfmq5\" (UID: \"a95cad0a-1c31-4ecf-915e-05b752bdaeee\") " pod="openshift-image-registry/node-ca-zfmq5" Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447282 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd1bec7b-f106-4ca2-b72d-71a7fe7a1929-hosts-file\") pod \"node-resolver-8z8zh\" (UID: \"bd1bec7b-f106-4ca2-b72d-71a7fe7a1929\") " pod="openshift-dns/node-resolver-8z8zh" Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447320 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/729bf7d2-7285-450d-8a22-f13dd26be9d8-cni-binary-copy\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447368 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/729bf7d2-7285-450d-8a22-f13dd26be9d8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447395 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-var-lib-kubelet\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447417 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-socket-dir\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447439 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39f4bef5-867c-4a17-abfe-489b216ffd8c-tmp\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447459 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-run-netns\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447485 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a3ae936-7af2-40dc-ac1b-88ed2341dbfe-agent-certs\") pod \"konnectivity-agent-2vmgg\" (UID: \"2a3ae936-7af2-40dc-ac1b-88ed2341dbfe\") " pod="kube-system/konnectivity-agent-2vmgg" Sep 10 22:01:12.447970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447509 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/729bf7d2-7285-450d-8a22-f13dd26be9d8-cnibin\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447532 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-dbus\") pod \"global-pull-secret-syncer-6dx6d\" (UID: \"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7\") " pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447554 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-etc-selinux\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447575 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-log-socket\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447599 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-cni-bin\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447616 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd1bec7b-f106-4ca2-b72d-71a7fe7a1929-tmp-dir\") pod \"node-resolver-8z8zh\" (UID: \"bd1bec7b-f106-4ca2-b72d-71a7fe7a1929\") " pod="openshift-dns/node-resolver-8z8zh" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447675 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/94294796-56b0-4302-8038-441734369d32-env-overrides\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447722 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a95cad0a-1c31-4ecf-915e-05b752bdaeee-serviceca\") pod \"node-ca-zfmq5\" (UID: \"a95cad0a-1c31-4ecf-915e-05b752bdaeee\") " pod="openshift-image-registry/node-ca-zfmq5" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447743 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-run-netns\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447777 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-dbus\") pod \"global-pull-secret-syncer-6dx6d\" (UID: \"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7\") " pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447789 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-socket-dir\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447819 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-etc-selinux\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447850 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/729bf7d2-7285-450d-8a22-f13dd26be9d8-cnibin\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447865 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-cni-bin\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447876 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-log-socket\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447622 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-cni-netd\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447910 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc6d7\" (UniqueName: \"kubernetes.io/projected/729bf7d2-7285-450d-8a22-f13dd26be9d8-kube-api-access-tc6d7\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447901 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-cni-netd\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.448742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447676 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-var-lib-kubelet\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447945 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-multus-cni-dir\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447966 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/729bf7d2-7285-450d-8a22-f13dd26be9d8-cni-binary-copy\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447979 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.447970 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-sys\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448010 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-sys\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448022 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-kubelet\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448048 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-node-log\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448059 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-kubelet\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448074 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/729bf7d2-7285-450d-8a22-f13dd26be9d8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448099 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/729bf7d2-7285-450d-8a22-f13dd26be9d8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448101 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-node-log\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448099 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-run-netns\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448216 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnn7w\" (UniqueName: \"kubernetes.io/projected/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-kube-api-access-hnn7w\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448229 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-host-run-netns\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448279 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-modprobe-d\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448307 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-sysctl-d\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448429 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-modprobe-d\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.449598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448477 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-systemd\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448522 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-multus-cni-dir\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448573 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-cni-binary-copy\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448614 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/729bf7d2-7285-450d-8a22-f13dd26be9d8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448654 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-systemd\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448686 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-multus-conf-dir\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448704 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-sysctl-d\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448713 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448737 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6-host-slash\") pod \"iptables-alerter-t7h7x\" (UID: \"cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6\") " pod="openshift-network-operator/iptables-alerter-t7h7x" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448744 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-multus-conf-dir\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448761 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-var-lib-openvswitch\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448789 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448789 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-multus-daemon-config\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448856 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret\") pod \"global-pull-secret-syncer-6dx6d\" (UID: \"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7\") " pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448881 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a3ae936-7af2-40dc-ac1b-88ed2341dbfe-konnectivity-ca\") pod \"konnectivity-agent-2vmgg\" (UID: \"2a3ae936-7af2-40dc-ac1b-88ed2341dbfe\") " pod="kube-system/konnectivity-agent-2vmgg" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448901 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-run-systemd\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448935 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-etc-openvswitch\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.450442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448958 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-run-ovn-kubernetes\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.448989 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-run-openvswitch\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449013 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/94294796-56b0-4302-8038-441734369d32-ovnkube-script-lib\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449036 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7r4v\" (UniqueName: \"kubernetes.io/projected/a95cad0a-1c31-4ecf-915e-05b752bdaeee-kube-api-access-w7r4v\") pod \"node-ca-zfmq5\" (UID: \"a95cad0a-1c31-4ecf-915e-05b752bdaeee\") " pod="openshift-image-registry/node-ca-zfmq5" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449057 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-device-dir\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449078 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6-iptables-alerter-script\") pod \"iptables-alerter-t7h7x\" (UID: \"cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6\") " pod="openshift-network-operator/iptables-alerter-t7h7x" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449100 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-kubernetes\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449122 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-run-ovn\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449144 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/94294796-56b0-4302-8038-441734369d32-ovnkube-config\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449173 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtw57\" (UniqueName: \"kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57\") pod \"network-check-target-kfbmj\" (UID: \"57428f1e-0c76-4457-937b-b33b503cb09a\") " pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449199 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/729bf7d2-7285-450d-8a22-f13dd26be9d8-system-cni-dir\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449223 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/729bf7d2-7285-450d-8a22-f13dd26be9d8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449247 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-hostroot\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:12.449250 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449255 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-multus-daemon-config\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449268 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-tuned\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449294 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.451085 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:12.449308 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret podName:291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7 nodeName:}" failed. No retries permitted until 2026-09-10 22:01:12.949290308 +0000 UTC m=+3.070989088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret") pod "global-pull-secret-syncer-6dx6d" (UID: "291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7") : object "kube-system"/"original-pull-secret" not registered Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449306 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6-host-slash\") pod \"iptables-alerter-t7h7x\" (UID: \"cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6\") " pod="openshift-network-operator/iptables-alerter-t7h7x" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449335 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-var-lib-openvswitch\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449378 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-run-systemd\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449412 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a3ae936-7af2-40dc-ac1b-88ed2341dbfe-konnectivity-ca\") pod \"konnectivity-agent-2vmgg\" (UID: \"2a3ae936-7af2-40dc-ac1b-88ed2341dbfe\") " pod="kube-system/konnectivity-agent-2vmgg" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449175 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-cni-binary-copy\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449650 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/729bf7d2-7285-450d-8a22-f13dd26be9d8-system-cni-dir\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449792 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/729bf7d2-7285-450d-8a22-f13dd26be9d8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449856 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-hostroot\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449863 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/94294796-56b0-4302-8038-441734369d32-ovnkube-script-lib\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449906 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6-iptables-alerter-script\") pod \"iptables-alerter-t7h7x\" (UID: \"cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6\") " pod="openshift-network-operator/iptables-alerter-t7h7x" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449911 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-etc-openvswitch\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449939 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-run-ovn-kubernetes\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449986 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-run-openvswitch\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.450034 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-kubernetes\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.450064 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-device-dir\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.449381 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-run-ovn\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.451744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.450066 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94294796-56b0-4302-8038-441734369d32-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.452543 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.450501 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/94294796-56b0-4302-8038-441734369d32-ovnkube-config\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.452578 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.452558 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/39f4bef5-867c-4a17-abfe-489b216ffd8c-etc-tuned\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.452755 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.452715 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39f4bef5-867c-4a17-abfe-489b216ffd8c-tmp\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.452899 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.452881 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/94294796-56b0-4302-8038-441734369d32-ovn-node-metrics-cert\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.452964 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.452902 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a3ae936-7af2-40dc-ac1b-88ed2341dbfe-agent-certs\") pod \"konnectivity-agent-2vmgg\" (UID: \"2a3ae936-7af2-40dc-ac1b-88ed2341dbfe\") " pod="kube-system/konnectivity-agent-2vmgg" Sep 10 22:01:12.459214 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:12.459188 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 22:01:12.459214 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:12.459217 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 22:01:12.459409 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:12.459242 2576 projected.go:196] Error preparing data for projected volume kube-api-access-gtw57 for pod openshift-network-diagnostics/network-check-target-kfbmj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:01:12.459409 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:12.459308 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57 podName:57428f1e-0c76-4457-937b-b33b503cb09a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:12.959287434 +0000 UTC m=+3.080986207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gtw57" (UniqueName: "kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57") pod "network-check-target-kfbmj" (UID: "57428f1e-0c76-4457-937b-b33b503cb09a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:01:12.460324 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.460288 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55cp2\" (UniqueName: \"kubernetes.io/projected/94294796-56b0-4302-8038-441734369d32-kube-api-access-55cp2\") pod \"ovnkube-node-sk86l\" (UID: \"94294796-56b0-4302-8038-441734369d32\") " pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.460448 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.460379 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjgkb\" (UniqueName: \"kubernetes.io/projected/39f4bef5-867c-4a17-abfe-489b216ffd8c-kube-api-access-sjgkb\") pod \"tuned-mbz8w\" (UID: \"39f4bef5-867c-4a17-abfe-489b216ffd8c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.461000 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.460936 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52hkf\" (UniqueName: \"kubernetes.io/projected/bd1bec7b-f106-4ca2-b72d-71a7fe7a1929-kube-api-access-52hkf\") pod \"node-resolver-8z8zh\" (UID: \"bd1bec7b-f106-4ca2-b72d-71a7fe7a1929\") " pod="openshift-dns/node-resolver-8z8zh" Sep 10 22:01:12.462147 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.461696 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzfpj\" (UniqueName: \"kubernetes.io/projected/cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6-kube-api-access-pzfpj\") pod \"iptables-alerter-t7h7x\" (UID: \"cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6\") " pod="openshift-network-operator/iptables-alerter-t7h7x" Sep 10 22:01:12.462147 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.462037 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx5x5\" (UniqueName: \"kubernetes.io/projected/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-kube-api-access-fx5x5\") pod \"network-metrics-daemon-lvd5t\" (UID: \"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0\") " pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:12.462662 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.462638 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc6d7\" (UniqueName: \"kubernetes.io/projected/729bf7d2-7285-450d-8a22-f13dd26be9d8-kube-api-access-tc6d7\") pod \"multus-additional-cni-plugins-pjbjk\" (UID: \"729bf7d2-7285-450d-8a22-f13dd26be9d8\") " pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.463147 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.463125 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnn7w\" (UniqueName: \"kubernetes.io/projected/e6e5a7c1-aff3-436f-87ed-cbda8019c06b-kube-api-access-hnn7w\") pod \"multus-bj2ck\" (UID: \"e6e5a7c1-aff3-436f-87ed-cbda8019c06b\") " pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.463768 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.463738 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7r4v\" (UniqueName: \"kubernetes.io/projected/a95cad0a-1c31-4ecf-915e-05b752bdaeee-kube-api-access-w7r4v\") pod \"node-ca-zfmq5\" (UID: \"a95cad0a-1c31-4ecf-915e-05b752bdaeee\") " pod="openshift-image-registry/node-ca-zfmq5" Sep 10 22:01:12.464710 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.464686 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gmc6\" (UniqueName: \"kubernetes.io/projected/1bb9fd9f-513b-44d8-9598-fd7bb57e1292-kube-api-access-6gmc6\") pod \"aws-ebs-csi-driver-node-ckjqx\" (UID: \"1bb9fd9f-513b-44d8-9598-fd7bb57e1292\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.629641 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.629595 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2vmgg" Sep 10 22:01:12.641913 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.641850 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t7h7x" Sep 10 22:01:12.642762 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:12.642731 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a3ae936_7af2_40dc_ac1b_88ed2341dbfe.slice/crio-397eaeaae7f9cb864622dde1f969aa2ebdf07b6e21a8cb060d7e95768dacf2ac WatchSource:0}: Error finding container 397eaeaae7f9cb864622dde1f969aa2ebdf07b6e21a8cb060d7e95768dacf2ac: Status 404 returned error can't find the container with id 397eaeaae7f9cb864622dde1f969aa2ebdf07b6e21a8cb060d7e95768dacf2ac Sep 10 22:01:12.651426 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.651394 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" Sep 10 22:01:12.651802 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:12.651762 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc7a2fd8_3f51_49eb_b8a6_23e37fda59a6.slice/crio-bf1aaf296036e3d113407bbf9cacea90f9188920e9de0b8a37d778e10627759c WatchSource:0}: Error finding container bf1aaf296036e3d113407bbf9cacea90f9188920e9de0b8a37d778e10627759c: Status 404 returned error can't find the container with id bf1aaf296036e3d113407bbf9cacea90f9188920e9de0b8a37d778e10627759c Sep 10 22:01:12.658188 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.658063 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:12.661278 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:12.661249 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39f4bef5_867c_4a17_abfe_489b216ffd8c.slice/crio-6c10f7d91ac7ee445581647498d2d5e7ed59cbd130bac49b215a0ccf33715a85 WatchSource:0}: Error finding container 6c10f7d91ac7ee445581647498d2d5e7ed59cbd130bac49b215a0ccf33715a85: Status 404 returned error can't find the container with id 6c10f7d91ac7ee445581647498d2d5e7ed59cbd130bac49b215a0ccf33715a85 Sep 10 22:01:12.665688 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.665604 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pjbjk" Sep 10 22:01:12.668124 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:12.668090 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94294796_56b0_4302_8038_441734369d32.slice/crio-58bdae8ecb0486a538a3eb04d34d05d796dd212b8d8e358efebcf9d624a565f7 WatchSource:0}: Error finding container 58bdae8ecb0486a538a3eb04d34d05d796dd212b8d8e358efebcf9d624a565f7: Status 404 returned error can't find the container with id 58bdae8ecb0486a538a3eb04d34d05d796dd212b8d8e358efebcf9d624a565f7 Sep 10 22:01:12.674763 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.674569 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bj2ck" Sep 10 22:01:12.675914 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:12.675864 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod729bf7d2_7285_450d_8a22_f13dd26be9d8.slice/crio-6044798115a9d4fb37cda5493f4bfb96ca9b8b29dbca968a8d894ac4ccc5a19c WatchSource:0}: Error finding container 6044798115a9d4fb37cda5493f4bfb96ca9b8b29dbca968a8d894ac4ccc5a19c: Status 404 returned error can't find the container with id 6044798115a9d4fb37cda5493f4bfb96ca9b8b29dbca968a8d894ac4ccc5a19c Sep 10 22:01:12.681791 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.681757 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8z8zh" Sep 10 22:01:12.684151 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:12.684032 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6e5a7c1_aff3_436f_87ed_cbda8019c06b.slice/crio-670e762962ebd91049a8b81cf4652d61113cc861ef4975d1a0b0d3a4349d2399 WatchSource:0}: Error finding container 670e762962ebd91049a8b81cf4652d61113cc861ef4975d1a0b0d3a4349d2399: Status 404 returned error can't find the container with id 670e762962ebd91049a8b81cf4652d61113cc861ef4975d1a0b0d3a4349d2399 Sep 10 22:01:12.689854 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.689825 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" Sep 10 22:01:12.691415 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:12.691383 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd1bec7b_f106_4ca2_b72d_71a7fe7a1929.slice/crio-18e5085f8607a9d83957332b12859d66b5062c2b3a8e28619a1dfd7271704afe WatchSource:0}: Error finding container 18e5085f8607a9d83957332b12859d66b5062c2b3a8e28619a1dfd7271704afe: Status 404 returned error can't find the container with id 18e5085f8607a9d83957332b12859d66b5062c2b3a8e28619a1dfd7271704afe Sep 10 22:01:12.695676 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.695642 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zfmq5" Sep 10 22:01:12.701065 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:12.701029 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bb9fd9f_513b_44d8_9598_fd7bb57e1292.slice/crio-d930bbf1f73c073e5c0b0f362ce947a5a6bec9b5883854ad2ff25a03b00c4899 WatchSource:0}: Error finding container d930bbf1f73c073e5c0b0f362ce947a5a6bec9b5883854ad2ff25a03b00c4899: Status 404 returned error can't find the container with id d930bbf1f73c073e5c0b0f362ce947a5a6bec9b5883854ad2ff25a03b00c4899 Sep 10 22:01:12.707359 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:12.707324 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda95cad0a_1c31_4ecf_915e_05b752bdaeee.slice/crio-afe695e56a93c22c07f2bb424a626f5668d6bf397fef37d4172a1eedb1568eee WatchSource:0}: Error finding container afe695e56a93c22c07f2bb424a626f5668d6bf397fef37d4172a1eedb1568eee: Status 404 returned error can't find the container with id afe695e56a93c22c07f2bb424a626f5668d6bf397fef37d4172a1eedb1568eee Sep 10 22:01:12.797260 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.797222 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 22:01:12.953793 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.953697 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs\") pod \"network-metrics-daemon-lvd5t\" (UID: \"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0\") " pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:12.953793 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:12.953783 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret\") pod \"global-pull-secret-syncer-6dx6d\" (UID: \"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7\") " pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:12.954042 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:12.953905 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:01:12.954042 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:12.953921 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 22:01:12.954042 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:12.953991 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret podName:291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7 nodeName:}" failed. No retries permitted until 2026-09-10 22:01:13.953970603 +0000 UTC m=+4.075669367 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret") pod "global-pull-secret-syncer-6dx6d" (UID: "291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7") : object "kube-system"/"original-pull-secret" not registered Sep 10 22:01:12.954042 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:12.954012 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs podName:6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0 nodeName:}" failed. No retries permitted until 2026-09-10 22:01:13.954002638 +0000 UTC m=+4.075701399 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs") pod "network-metrics-daemon-lvd5t" (UID: "6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:01:13.054938 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:13.054884 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtw57\" (UniqueName: \"kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57\") pod \"network-check-target-kfbmj\" (UID: \"57428f1e-0c76-4457-937b-b33b503cb09a\") " pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:13.055160 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:13.055133 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 22:01:13.055160 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:13.055162 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 22:01:13.055337 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:13.055175 2576 projected.go:196] Error preparing data for projected volume kube-api-access-gtw57 for pod openshift-network-diagnostics/network-check-target-kfbmj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:01:13.055337 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:13.055247 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57 podName:57428f1e-0c76-4457-937b-b33b503cb09a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:14.055227109 +0000 UTC m=+4.176925874 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gtw57" (UniqueName: "kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57") pod "network-check-target-kfbmj" (UID: "57428f1e-0c76-4457-937b-b33b503cb09a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:01:13.391049 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:13.390994 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 21:56:12 +0000 UTC" deadline="2028-05-11 08:15:11.881867171 +0000 UTC" Sep 10 22:01:13.391049 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:13.391042 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14602h13m58.490830621s" Sep 10 22:01:13.446741 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:13.446694 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zfmq5" event={"ID":"a95cad0a-1c31-4ecf-915e-05b752bdaeee","Type":"ContainerStarted","Data":"afe695e56a93c22c07f2bb424a626f5668d6bf397fef37d4172a1eedb1568eee"} Sep 10 22:01:13.452967 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:13.452880 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" event={"ID":"1bb9fd9f-513b-44d8-9598-fd7bb57e1292","Type":"ContainerStarted","Data":"d930bbf1f73c073e5c0b0f362ce947a5a6bec9b5883854ad2ff25a03b00c4899"} Sep 10 22:01:13.456845 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:13.456624 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" event={"ID":"94294796-56b0-4302-8038-441734369d32","Type":"ContainerStarted","Data":"58bdae8ecb0486a538a3eb04d34d05d796dd212b8d8e358efebcf9d624a565f7"} Sep 10 22:01:13.461263 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:13.461230 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t7h7x" event={"ID":"cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6","Type":"ContainerStarted","Data":"bf1aaf296036e3d113407bbf9cacea90f9188920e9de0b8a37d778e10627759c"} Sep 10 22:01:13.467680 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:13.467593 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2vmgg" event={"ID":"2a3ae936-7af2-40dc-ac1b-88ed2341dbfe","Type":"ContainerStarted","Data":"397eaeaae7f9cb864622dde1f969aa2ebdf07b6e21a8cb060d7e95768dacf2ac"} Sep 10 22:01:13.471351 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:13.471265 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8z8zh" event={"ID":"bd1bec7b-f106-4ca2-b72d-71a7fe7a1929","Type":"ContainerStarted","Data":"18e5085f8607a9d83957332b12859d66b5062c2b3a8e28619a1dfd7271704afe"} Sep 10 22:01:13.478781 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:13.478686 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bj2ck" event={"ID":"e6e5a7c1-aff3-436f-87ed-cbda8019c06b","Type":"ContainerStarted","Data":"670e762962ebd91049a8b81cf4652d61113cc861ef4975d1a0b0d3a4349d2399"} Sep 10 22:01:13.482709 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:13.482645 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjbjk" event={"ID":"729bf7d2-7285-450d-8a22-f13dd26be9d8","Type":"ContainerStarted","Data":"6044798115a9d4fb37cda5493f4bfb96ca9b8b29dbca968a8d894ac4ccc5a19c"} Sep 10 22:01:13.487003 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:13.486920 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" event={"ID":"39f4bef5-867c-4a17-abfe-489b216ffd8c","Type":"ContainerStarted","Data":"6c10f7d91ac7ee445581647498d2d5e7ed59cbd130bac49b215a0ccf33715a85"} Sep 10 22:01:13.963267 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:13.963177 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret\") pod \"global-pull-secret-syncer-6dx6d\" (UID: \"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7\") " pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:13.963267 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:13.963272 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs\") pod \"network-metrics-daemon-lvd5t\" (UID: \"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0\") " pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:13.963547 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:13.963414 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:01:13.963547 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:13.963476 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs podName:6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0 nodeName:}" failed. No retries permitted until 2026-09-10 22:01:15.963457632 +0000 UTC m=+6.085156405 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs") pod "network-metrics-daemon-lvd5t" (UID: "6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:01:13.963955 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:13.963936 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 22:01:13.964026 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:13.963995 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret podName:291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7 nodeName:}" failed. No retries permitted until 2026-09-10 22:01:15.963979338 +0000 UTC m=+6.085678101 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret") pod "global-pull-secret-syncer-6dx6d" (UID: "291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7") : object "kube-system"/"original-pull-secret" not registered Sep 10 22:01:14.064467 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:14.063829 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtw57\" (UniqueName: \"kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57\") pod \"network-check-target-kfbmj\" (UID: \"57428f1e-0c76-4457-937b-b33b503cb09a\") " pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:14.064467 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:14.064026 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 22:01:14.064467 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:14.064049 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 22:01:14.064467 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:14.064062 2576 projected.go:196] Error preparing data for projected volume kube-api-access-gtw57 for pod openshift-network-diagnostics/network-check-target-kfbmj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:01:14.064467 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:14.064123 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57 podName:57428f1e-0c76-4457-937b-b33b503cb09a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:16.06410496 +0000 UTC m=+6.185803745 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gtw57" (UniqueName: "kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57") pod "network-check-target-kfbmj" (UID: "57428f1e-0c76-4457-937b-b33b503cb09a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:01:14.391537 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:14.391440 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 21:56:12 +0000 UTC" deadline="2028-02-08 09:42:10.994283986 +0000 UTC" Sep 10 22:01:14.391537 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:14.391510 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12371h40m56.602778748s" Sep 10 22:01:14.428260 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:14.427889 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:14.428260 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:14.428002 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:14.428260 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:14.428083 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:14.428260 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:14.428104 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:14.428260 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:14.428161 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:14.428260 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:14.428226 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:14.495736 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:14.495699 2576 generic.go:356] "Generic (PLEG): container finished" podID="5f12632b0c737aeacd907582d37aedae" containerID="97af430c0e0e93d44e1a9baad565491e95bddeadfe3ea752c9d953fbbd9cceed" exitCode=0 Sep 10 22:01:14.495971 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:14.495763 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal" event={"ID":"5f12632b0c737aeacd907582d37aedae","Type":"ContainerDied","Data":"97af430c0e0e93d44e1a9baad565491e95bddeadfe3ea752c9d953fbbd9cceed"} Sep 10 22:01:15.979457 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:15.979422 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs\") pod \"network-metrics-daemon-lvd5t\" (UID: \"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0\") " pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:15.980053 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:15.979496 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret\") pod \"global-pull-secret-syncer-6dx6d\" (UID: \"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7\") " pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:15.980053 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:15.979613 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:01:15.980053 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:15.979642 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 22:01:15.980053 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:15.979678 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs podName:6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0 nodeName:}" failed. No retries permitted until 2026-09-10 22:01:19.979654341 +0000 UTC m=+10.101353102 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs") pod "network-metrics-daemon-lvd5t" (UID: "6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:01:15.980053 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:15.979696 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret podName:291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7 nodeName:}" failed. No retries permitted until 2026-09-10 22:01:19.979687676 +0000 UTC m=+10.101386436 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret") pod "global-pull-secret-syncer-6dx6d" (UID: "291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7") : object "kube-system"/"original-pull-secret" not registered Sep 10 22:01:16.080145 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:16.080098 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtw57\" (UniqueName: \"kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57\") pod \"network-check-target-kfbmj\" (UID: \"57428f1e-0c76-4457-937b-b33b503cb09a\") " pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:16.080305 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:16.080288 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 22:01:16.080305 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:16.080304 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 22:01:16.080387 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:16.080313 2576 projected.go:196] Error preparing data for projected volume kube-api-access-gtw57 for pod openshift-network-diagnostics/network-check-target-kfbmj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:01:16.080387 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:16.080375 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57 podName:57428f1e-0c76-4457-937b-b33b503cb09a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:20.080356347 +0000 UTC m=+10.202055127 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gtw57" (UniqueName: "kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57") pod "network-check-target-kfbmj" (UID: "57428f1e-0c76-4457-937b-b33b503cb09a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:01:16.426938 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:16.426908 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:16.427103 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:16.427031 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:16.427435 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:16.427413 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:16.427572 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:16.427521 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:16.427694 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:16.427673 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:16.427789 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:16.427767 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:18.425861 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:18.425642 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:18.425861 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:18.425642 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:18.425861 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:18.425812 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:18.426424 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:18.425887 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:18.426424 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:18.425642 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:18.426424 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:18.425979 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:20.015862 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:20.015820 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs\") pod \"network-metrics-daemon-lvd5t\" (UID: \"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0\") " pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:20.016335 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:20.015896 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret\") pod \"global-pull-secret-syncer-6dx6d\" (UID: \"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7\") " pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:20.016335 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:20.016024 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 22:01:20.016335 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:20.016090 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret podName:291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7 nodeName:}" failed. No retries permitted until 2026-09-10 22:01:28.016071725 +0000 UTC m=+18.137770503 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret") pod "global-pull-secret-syncer-6dx6d" (UID: "291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7") : object "kube-system"/"original-pull-secret" not registered Sep 10 22:01:20.016335 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:20.016260 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:01:20.016335 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:20.016324 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs podName:6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0 nodeName:}" failed. No retries permitted until 2026-09-10 22:01:28.016308337 +0000 UTC m=+18.138007110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs") pod "network-metrics-daemon-lvd5t" (UID: "6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:01:20.116579 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:20.116541 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtw57\" (UniqueName: \"kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57\") pod \"network-check-target-kfbmj\" (UID: \"57428f1e-0c76-4457-937b-b33b503cb09a\") " pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:20.116773 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:20.116748 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 22:01:20.116773 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:20.116774 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 22:01:20.116918 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:20.116788 2576 projected.go:196] Error preparing data for projected volume kube-api-access-gtw57 for pod openshift-network-diagnostics/network-check-target-kfbmj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:01:20.116918 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:20.116873 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57 podName:57428f1e-0c76-4457-937b-b33b503cb09a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:28.116852251 +0000 UTC m=+18.238551028 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gtw57" (UniqueName: "kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57") pod "network-check-target-kfbmj" (UID: "57428f1e-0c76-4457-937b-b33b503cb09a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:01:20.425958 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:20.425917 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:20.426156 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:20.426051 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:20.426695 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:20.426368 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:20.426695 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:20.426476 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:20.426695 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:20.426540 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:20.426695 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:20.426610 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:22.425963 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:22.425926 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:22.426378 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:22.425926 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:22.426378 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:22.426081 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:22.426378 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:22.426146 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:22.426378 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:22.425927 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:22.426378 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:22.426240 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:24.425686 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:24.425642 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:24.425686 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:24.425681 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:24.426177 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:24.425642 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:24.426177 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:24.425812 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:24.426177 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:24.425887 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:24.426177 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:24.425964 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:26.428777 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:26.428740 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:26.429331 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:26.428782 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:26.429331 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:26.428877 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:26.429331 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:26.428896 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:26.429331 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:26.428979 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:26.429331 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:26.429069 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:28.075078 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:28.075037 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs\") pod \"network-metrics-daemon-lvd5t\" (UID: \"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0\") " pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:28.075597 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:28.075110 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret\") pod \"global-pull-secret-syncer-6dx6d\" (UID: \"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7\") " pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:28.075597 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:28.075236 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 22:01:28.075597 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:28.075260 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:01:28.075597 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:28.075300 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret podName:291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7 nodeName:}" failed. No retries permitted until 2026-09-10 22:01:44.075284503 +0000 UTC m=+34.196983280 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret") pod "global-pull-secret-syncer-6dx6d" (UID: "291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7") : object "kube-system"/"original-pull-secret" not registered Sep 10 22:01:28.075597 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:28.075325 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs podName:6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0 nodeName:}" failed. No retries permitted until 2026-09-10 22:01:44.0753067 +0000 UTC m=+34.197005484 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs") pod "network-metrics-daemon-lvd5t" (UID: "6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:01:28.176152 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:28.176106 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtw57\" (UniqueName: \"kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57\") pod \"network-check-target-kfbmj\" (UID: \"57428f1e-0c76-4457-937b-b33b503cb09a\") " pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:28.176325 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:28.176294 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 22:01:28.176325 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:28.176318 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 22:01:28.176426 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:28.176328 2576 projected.go:196] Error preparing data for projected volume kube-api-access-gtw57 for pod openshift-network-diagnostics/network-check-target-kfbmj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:01:28.176426 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:28.176391 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57 podName:57428f1e-0c76-4457-937b-b33b503cb09a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:44.176374433 +0000 UTC m=+34.298073213 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gtw57" (UniqueName: "kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57") pod "network-check-target-kfbmj" (UID: "57428f1e-0c76-4457-937b-b33b503cb09a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:01:28.425468 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:28.425378 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:28.425468 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:28.425439 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:28.425730 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:28.425378 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:28.425730 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:28.425531 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:28.425730 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:28.425628 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:28.425893 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:28.425785 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:30.425913 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:30.425884 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:30.426299 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:30.426006 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:30.426299 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:30.426044 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:30.426299 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:30.426071 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:30.426299 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:30.426145 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:30.426299 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:30.426244 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:30.567048 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:30.564415 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-247.ec2.internal" event={"ID":"6969b8244f652594d09b7861b9d8eb44","Type":"ContainerStarted","Data":"d3fc27e2be7e09d1e7076582999a242df1359c96aee07529d861351a5eec5746"} Sep 10 22:01:30.604060 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:30.603403 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-247.ec2.internal" podStartSLOduration=19.603379225 podStartE2EDuration="19.603379225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:01:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:01:30.579842407 +0000 UTC m=+20.701541185" watchObservedRunningTime="2026-09-10 22:01:30.603379225 +0000 UTC m=+20.725078008" Sep 10 22:01:30.604060 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:30.603596 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" podStartSLOduration=2.8625529419999998 podStartE2EDuration="20.603587219s" totalImagesPullingTime="17.741034277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:01:30.600435232 +0000 UTC m=+20.722134016" watchObservedRunningTime="2026-09-10 22:01:30.603587219 +0000 UTC m=+20.725286004" Sep 10 22:01:30.638991 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:30.638941 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2vmgg" podStartSLOduration=10.096415529 podStartE2EDuration="20.638917336s" totalImagesPullingTime="10.542501807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:01:30.638665948 +0000 UTC m=+20.760364731" watchObservedRunningTime="2026-09-10 22:01:30.638917336 +0000 UTC m=+20.760616118" Sep 10 22:01:30.639118 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:30.639094 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal" podStartSLOduration=19.639086207 podStartE2EDuration="19.639086207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:01:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:01:30.620239086 +0000 UTC m=+20.741937869" watchObservedRunningTime="2026-09-10 22:01:30.639086207 +0000 UTC m=+20.760784990" Sep 10 22:01:31.612792 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.612525 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8z8zh" event={"ID":"bd1bec7b-f106-4ca2-b72d-71a7fe7a1929","Type":"ContainerStarted","Data":"f8d9497ca10dd5276792235cf257807aac713fdce0d9992cf40ff1bc50b37ca2"} Sep 10 22:01:31.614458 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.614425 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bj2ck" event={"ID":"e6e5a7c1-aff3-436f-87ed-cbda8019c06b","Type":"ContainerStarted","Data":"7fb17a9099341090680f4edbc72f143a9107d9866504f03ce3f59d01f7c3d342"} Sep 10 22:01:31.616354 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.616327 2576 generic.go:356] "Generic (PLEG): container finished" podID="729bf7d2-7285-450d-8a22-f13dd26be9d8" containerID="d0580c4ad27193cc10d607372a627d1784892bb0596a938c4f8eb61512d43cd1" exitCode=0 Sep 10 22:01:31.616499 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.616418 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjbjk" event={"ID":"729bf7d2-7285-450d-8a22-f13dd26be9d8","Type":"ContainerDied","Data":"d0580c4ad27193cc10d607372a627d1784892bb0596a938c4f8eb61512d43cd1"} Sep 10 22:01:31.618339 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.618310 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mbz8w" event={"ID":"39f4bef5-867c-4a17-abfe-489b216ffd8c","Type":"ContainerStarted","Data":"f26d970018ac720b9983efc18ed277685ef22c63c8909debd9388b899bfb20bd"} Sep 10 22:01:31.620590 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.620564 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-247.ec2.internal" event={"ID":"5f12632b0c737aeacd907582d37aedae","Type":"ContainerStarted","Data":"f069cb0342cb6689e4b3ca2ed59bfd5a966abceae1d359d3daf1f19adc24934a"} Sep 10 22:01:31.622185 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.622165 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zfmq5" event={"ID":"a95cad0a-1c31-4ecf-915e-05b752bdaeee","Type":"ContainerStarted","Data":"9a54d23da38da8b1629cdff6cd2fcc3e71e1a3263f2044108dcba00a351a1547"} Sep 10 22:01:31.623928 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.623872 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" event={"ID":"1bb9fd9f-513b-44d8-9598-fd7bb57e1292","Type":"ContainerStarted","Data":"9fd1c61902c0436528b7876471bc9248117bafa6d5b45cb0a9474248652f0aca"} Sep 10 22:01:31.627357 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.627334 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" event={"ID":"94294796-56b0-4302-8038-441734369d32","Type":"ContainerStarted","Data":"23c23a37dbceff33b4006b024a14c432c5c1b7a2e600cd981e34bef857be3bf3"} Sep 10 22:01:31.627491 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.627406 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" event={"ID":"94294796-56b0-4302-8038-441734369d32","Type":"ContainerStarted","Data":"ca2f06704dbbacca9c9e083c05f190f656ea137f831752630a650429f4da4222"} Sep 10 22:01:31.627491 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.627449 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" event={"ID":"94294796-56b0-4302-8038-441734369d32","Type":"ContainerStarted","Data":"0cc1578fd5686b242a9a94143410bd37513da57355c0b507a206f3a7175492bb"} Sep 10 22:01:31.627491 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.627484 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" event={"ID":"94294796-56b0-4302-8038-441734369d32","Type":"ContainerStarted","Data":"0635b6e9bce9b45403815b6eb48ff520d5dde38a6ecb11814e952f3f8400ea6b"} Sep 10 22:01:31.627646 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.627518 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" event={"ID":"94294796-56b0-4302-8038-441734369d32","Type":"ContainerStarted","Data":"af3ccf66f798ec4ba53859f6724ce0faab9810785ab0245e56dc79ecedc2691b"} Sep 10 22:01:31.627646 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.627565 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" event={"ID":"94294796-56b0-4302-8038-441734369d32","Type":"ContainerStarted","Data":"e7980cfa279041d5afe6fd577032cbc63e49403df5045bdc20ac5033ad3f8aad"} Sep 10 22:01:31.627922 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.627880 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8z8zh" podStartSLOduration=3.903749339 podStartE2EDuration="21.627863003s" totalImagesPullingTime="17.724113664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:01:31.627604367 +0000 UTC m=+21.749303172" watchObservedRunningTime="2026-09-10 22:01:31.627863003 +0000 UTC m=+21.749561788" Sep 10 22:01:31.628924 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.628902 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2vmgg" event={"ID":"2a3ae936-7af2-40dc-ac1b-88ed2341dbfe","Type":"ContainerStarted","Data":"81acc5b95ff6802e56bd51462521ef98032498bdc926d6790ab868257fe75237"} Sep 10 22:01:31.662260 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.662201 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zfmq5" podStartSLOduration=3.201000418 podStartE2EDuration="20.662182643s" totalImagesPullingTime="17.461182225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:01:31.661550913 +0000 UTC m=+21.783249696" watchObservedRunningTime="2026-09-10 22:01:31.662182643 +0000 UTC m=+21.783881437" Sep 10 22:01:31.678640 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.678588 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bj2ck" podStartSLOduration=3.943022491 podStartE2EDuration="21.678550396s" totalImagesPullingTime="17.735527905s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:01:31.678176611 +0000 UTC m=+21.799875419" watchObservedRunningTime="2026-09-10 22:01:31.678550396 +0000 UTC m=+21.800249178" Sep 10 22:01:31.879532 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:31.879503 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 22:01:32.408615 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:32.408474 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T22:01:31.87952629Z","UUID":"773dc7a1-d448-4a67-af72-a91835e5cec0","Handler":null,"Name":"","Endpoint":""} Sep 10 22:01:32.410261 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:32.410230 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 22:01:32.410406 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:32.410270 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 22:01:32.425347 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:32.425313 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:32.425542 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:32.425313 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:32.425542 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:32.425430 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:32.425542 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:32.425453 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:32.425704 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:32.425561 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:32.425704 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:32.425657 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:32.633817 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:32.633759 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" event={"ID":"1bb9fd9f-513b-44d8-9598-fd7bb57e1292","Type":"ContainerStarted","Data":"96546a4808b9f1e28630a1aed9b1f52f984da333a4c73d57deedfef6f3c41ae9"} Sep 10 22:01:32.635505 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:32.635471 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t7h7x" event={"ID":"cc7a2fd8-3f51-49eb-b8a6-23e37fda59a6","Type":"ContainerStarted","Data":"a127346fc9ce05c5139e1c1e6e268f689946eaeef2f795fc3eaad86e532cccb8"} Sep 10 22:01:32.651590 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:32.651535 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-t7h7x" podStartSLOduration=5.134197128 podStartE2EDuration="22.651515771s" totalImagesPullingTime="17.517318643s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:01:32.651365573 +0000 UTC m=+22.773064351" watchObservedRunningTime="2026-09-10 22:01:32.651515771 +0000 UTC m=+22.773214553" Sep 10 22:01:33.640790 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:33.640525 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" event={"ID":"1bb9fd9f-513b-44d8-9598-fd7bb57e1292","Type":"ContainerStarted","Data":"1ff475573ce74867f3e03f2030fcc8d4541f63bdb2118ef3af6f8aa51a4690a1"} Sep 10 22:01:33.644893 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:33.644866 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" event={"ID":"94294796-56b0-4302-8038-441734369d32","Type":"ContainerStarted","Data":"56854c70f97559c370efd8f6928861cfe8b1824ddf0d6c08c90b94625434db68"} Sep 10 22:01:33.658555 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:33.658506 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckjqx" podStartSLOduration=2.867426283 podStartE2EDuration="22.658490582s" totalImagesPullingTime="19.791064299s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:01:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:01:33.65810983 +0000 UTC m=+23.779808612" watchObservedRunningTime="2026-09-10 22:01:33.658490582 +0000 UTC m=+23.780189364" Sep 10 22:01:34.426214 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:34.426176 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:34.426409 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:34.426228 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:34.426409 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:34.426176 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:34.426409 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:34.426315 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:34.426574 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:34.426411 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:34.426574 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:34.426525 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:34.569148 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:34.569111 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2vmgg" Sep 10 22:01:34.569870 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:34.569841 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2vmgg" Sep 10 22:01:34.647134 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:34.647108 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2vmgg" Sep 10 22:01:34.647513 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:34.647298 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2vmgg" Sep 10 22:01:36.426197 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:36.426156 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:36.426645 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:36.426156 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:36.426645 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:36.426305 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:36.426645 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:36.426386 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:36.426645 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:36.426169 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:36.426645 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:36.426483 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:36.653614 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:36.653396 2576 generic.go:356] "Generic (PLEG): container finished" podID="729bf7d2-7285-450d-8a22-f13dd26be9d8" containerID="01342b8e859e8e291046699150e8e4b4f82a031401ee90e0b1da29284d5554c4" exitCode=0 Sep 10 22:01:36.653814 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:36.653475 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjbjk" event={"ID":"729bf7d2-7285-450d-8a22-f13dd26be9d8","Type":"ContainerDied","Data":"01342b8e859e8e291046699150e8e4b4f82a031401ee90e0b1da29284d5554c4"} Sep 10 22:01:36.657886 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:36.657853 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" event={"ID":"94294796-56b0-4302-8038-441734369d32","Type":"ContainerStarted","Data":"910823b36a4f2b2bd07c600ac7d7be0a597c288c7e5dfcbd55e848b86f9b42b2"} Sep 10 22:01:36.658093 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:36.658074 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:36.673681 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:36.673656 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:36.696427 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:36.696366 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" podStartSLOduration=8.790604375000001 podStartE2EDuration="26.696350721s" totalImagesPullingTime="17.905746346s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:01:36.695442176 +0000 UTC m=+26.817140952" watchObservedRunningTime="2026-09-10 22:01:36.696350721 +0000 UTC m=+26.818049503" Sep 10 22:01:37.660614 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:37.660581 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:37.661186 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:37.660671 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:37.675401 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:37.675375 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:01:38.176827 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:38.176775 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6dx6d"] Sep 10 22:01:38.176998 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:38.176954 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:38.177065 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:38.177043 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:38.180584 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:38.180555 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kfbmj"] Sep 10 22:01:38.180726 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:38.180685 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:38.180831 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:38.180791 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:38.194641 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:38.194604 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lvd5t"] Sep 10 22:01:38.194821 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:38.194744 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:38.194889 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:38.194869 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:38.665089 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:38.665045 2576 generic.go:356] "Generic (PLEG): container finished" podID="729bf7d2-7285-450d-8a22-f13dd26be9d8" containerID="d2ac2336135c92b3bf1dc856c54366b2f3c5f24404ef3c850e21ab25d18cdb5b" exitCode=0 Sep 10 22:01:38.665788 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:38.665129 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjbjk" event={"ID":"729bf7d2-7285-450d-8a22-f13dd26be9d8","Type":"ContainerDied","Data":"d2ac2336135c92b3bf1dc856c54366b2f3c5f24404ef3c850e21ab25d18cdb5b"} Sep 10 22:01:39.425883 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:39.425845 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:39.426045 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:39.425998 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:39.670079 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:39.670046 2576 generic.go:356] "Generic (PLEG): container finished" podID="729bf7d2-7285-450d-8a22-f13dd26be9d8" containerID="9e8375994ba6bfb2316645bfa74e94b1bb9202ecc984279c9e559c1172268cbb" exitCode=0 Sep 10 22:01:39.670515 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:39.670124 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjbjk" event={"ID":"729bf7d2-7285-450d-8a22-f13dd26be9d8","Type":"ContainerDied","Data":"9e8375994ba6bfb2316645bfa74e94b1bb9202ecc984279c9e559c1172268cbb"} Sep 10 22:01:40.426007 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:40.425828 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:40.426201 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:40.425879 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:40.426201 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:40.426065 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:40.426201 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:40.426168 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:41.426188 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:41.426150 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:41.426658 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:41.426286 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dx6d" podUID="291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7" Sep 10 22:01:42.426012 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:42.425974 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:42.426273 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:42.426087 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kfbmj" podUID="57428f1e-0c76-4457-937b-b33b503cb09a" Sep 10 22:01:42.426273 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:42.426157 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:42.426685 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:42.426297 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvd5t" podUID="6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0" Sep 10 22:01:43.157553 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.157503 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-247.ec2.internal" event="NodeReady" Sep 10 22:01:43.157775 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.157761 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 22:01:43.196426 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.196388 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cdb65cf65-bnlmv"] Sep 10 22:01:43.225707 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.225665 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cdb65cf65-bnlmv"] Sep 10 22:01:43.225707 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.225720 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hmlp2"] Sep 10 22:01:43.225962 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.225755 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.228258 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.228212 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 10 22:01:43.228687 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.228666 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 10 22:01:43.229083 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.228909 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 10 22:01:43.229315 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.229233 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c5lwm\"" Sep 10 22:01:43.234519 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.234385 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 10 22:01:43.241858 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.241830 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7tzf6"] Sep 10 22:01:43.241995 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.241903 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:43.243939 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.243917 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 22:01:43.244079 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.243971 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 22:01:43.244079 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.243990 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qt6wb\"" Sep 10 22:01:43.263530 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.263494 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hmlp2"] Sep 10 22:01:43.263530 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.263541 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7tzf6"] Sep 10 22:01:43.263762 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.263605 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7tzf6" Sep 10 22:01:43.266413 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.266388 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 22:01:43.266561 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.266439 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 22:01:43.266561 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.266453 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 22:01:43.266561 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.266509 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fw62z\"" Sep 10 22:01:43.293542 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.293506 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/31f47884-061b-4eb7-b27b-91d855d9bf3a-ca-trust-extracted\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.293736 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.293574 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/31f47884-061b-4eb7-b27b-91d855d9bf3a-trusted-ca\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.293736 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.293622 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/31f47884-061b-4eb7-b27b-91d855d9bf3a-installation-pull-secrets\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.293736 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.293663 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/31f47884-061b-4eb7-b27b-91d855d9bf3a-image-registry-private-configuration\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.293736 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.293685 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-bound-sa-token\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.293736 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.293704 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6bbq\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-kube-api-access-d6bbq\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.294010 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.293744 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.294010 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.293810 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-certificates\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.377261 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.377223 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:43.394168 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.394131 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9294b995-aea4-496b-8e2c-9739ceda5668-cert\") pod \"ingress-canary-7tzf6\" (UID: \"9294b995-aea4-496b-8e2c-9739ceda5668\") " pod="openshift-ingress-canary/ingress-canary-7tzf6" Sep 10 22:01:43.394168 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.394174 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/31f47884-061b-4eb7-b27b-91d855d9bf3a-trusted-ca\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.394382 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.394196 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d7e6f82c-4cc2-4610-892d-9e2581c1264a-config-volume\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:43.394382 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.394221 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e6f82c-4cc2-4610-892d-9e2581c1264a-tmp-dir\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:43.394382 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.394358 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/31f47884-061b-4eb7-b27b-91d855d9bf3a-installation-pull-secrets\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.394528 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.394414 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz89h\" (UniqueName: \"kubernetes.io/projected/9294b995-aea4-496b-8e2c-9739ceda5668-kube-api-access-lz89h\") pod \"ingress-canary-7tzf6\" (UID: \"9294b995-aea4-496b-8e2c-9739ceda5668\") " pod="openshift-ingress-canary/ingress-canary-7tzf6" Sep 10 22:01:43.394528 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.394444 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/31f47884-061b-4eb7-b27b-91d855d9bf3a-image-registry-private-configuration\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.394528 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.394474 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-bound-sa-token\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.394528 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.394505 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6bbq\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-kube-api-access-d6bbq\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.394763 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.394546 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.394763 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.394575 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-certificates\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.394763 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.394602 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:43.394763 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.394625 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6rc2\" (UniqueName: \"kubernetes.io/projected/d7e6f82c-4cc2-4610-892d-9e2581c1264a-kube-api-access-f6rc2\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:43.394763 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.394661 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/31f47884-061b-4eb7-b27b-91d855d9bf3a-ca-trust-extracted\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.394763 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:43.394693 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 22:01:43.394763 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:43.394708 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cdb65cf65-bnlmv: secret "image-registry-tls" not found Sep 10 22:01:43.395134 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:43.394780 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls podName:31f47884-061b-4eb7-b27b-91d855d9bf3a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:43.894761421 +0000 UTC m=+34.016460194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls") pod "image-registry-6cdb65cf65-bnlmv" (UID: "31f47884-061b-4eb7-b27b-91d855d9bf3a") : secret "image-registry-tls" not found Sep 10 22:01:43.395134 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.395090 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/31f47884-061b-4eb7-b27b-91d855d9bf3a-ca-trust-extracted\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.395290 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.395257 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/31f47884-061b-4eb7-b27b-91d855d9bf3a-trusted-ca\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.395408 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.395319 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-certificates\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.399172 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.399144 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/31f47884-061b-4eb7-b27b-91d855d9bf3a-installation-pull-secrets\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.399311 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.399145 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/31f47884-061b-4eb7-b27b-91d855d9bf3a-image-registry-private-configuration\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.405404 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.405372 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-bound-sa-token\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.406369 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.406340 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6bbq\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-kube-api-access-d6bbq\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.425530 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.425497 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:43.428147 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.428124 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 22:01:43.478621 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.478586 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:43.495721 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.495627 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9294b995-aea4-496b-8e2c-9739ceda5668-cert\") pod \"ingress-canary-7tzf6\" (UID: \"9294b995-aea4-496b-8e2c-9739ceda5668\") " pod="openshift-ingress-canary/ingress-canary-7tzf6" Sep 10 22:01:43.495721 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.495671 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d7e6f82c-4cc2-4610-892d-9e2581c1264a-config-volume\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:43.495721 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.495691 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e6f82c-4cc2-4610-892d-9e2581c1264a-tmp-dir\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:43.496024 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.495736 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz89h\" (UniqueName: \"kubernetes.io/projected/9294b995-aea4-496b-8e2c-9739ceda5668-kube-api-access-lz89h\") pod \"ingress-canary-7tzf6\" (UID: \"9294b995-aea4-496b-8e2c-9739ceda5668\") " pod="openshift-ingress-canary/ingress-canary-7tzf6" Sep 10 22:01:43.496024 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.495794 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:43.496024 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.495835 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6rc2\" (UniqueName: \"kubernetes.io/projected/d7e6f82c-4cc2-4610-892d-9e2581c1264a-kube-api-access-f6rc2\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:43.496024 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:43.495913 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 22:01:43.496024 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:43.495976 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls podName:d7e6f82c-4cc2-4610-892d-9e2581c1264a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:43.995961734 +0000 UTC m=+34.117660506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls") pod "dns-default-hmlp2" (UID: "d7e6f82c-4cc2-4610-892d-9e2581c1264a") : secret "dns-default-metrics-tls" not found Sep 10 22:01:43.496283 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.496114 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e6f82c-4cc2-4610-892d-9e2581c1264a-tmp-dir\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:43.496357 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.496332 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d7e6f82c-4cc2-4610-892d-9e2581c1264a-config-volume\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:43.498718 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.498696 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9294b995-aea4-496b-8e2c-9739ceda5668-cert\") pod \"ingress-canary-7tzf6\" (UID: \"9294b995-aea4-496b-8e2c-9739ceda5668\") " pod="openshift-ingress-canary/ingress-canary-7tzf6" Sep 10 22:01:43.504174 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.504140 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6rc2\" (UniqueName: \"kubernetes.io/projected/d7e6f82c-4cc2-4610-892d-9e2581c1264a-kube-api-access-f6rc2\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:43.504322 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.504303 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz89h\" (UniqueName: \"kubernetes.io/projected/9294b995-aea4-496b-8e2c-9739ceda5668-kube-api-access-lz89h\") pod \"ingress-canary-7tzf6\" (UID: \"9294b995-aea4-496b-8e2c-9739ceda5668\") " pod="openshift-ingress-canary/ingress-canary-7tzf6" Sep 10 22:01:43.574463 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.574425 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7tzf6" Sep 10 22:01:43.898859 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.898820 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:43.899081 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:43.899000 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 22:01:43.899081 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:43.899024 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cdb65cf65-bnlmv: secret "image-registry-tls" not found Sep 10 22:01:43.899181 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:43.899086 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls podName:31f47884-061b-4eb7-b27b-91d855d9bf3a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:44.899069126 +0000 UTC m=+35.020767886 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls") pod "image-registry-6cdb65cf65-bnlmv" (UID: "31f47884-061b-4eb7-b27b-91d855d9bf3a") : secret "image-registry-tls" not found Sep 10 22:01:43.999522 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:43.999485 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:43.999740 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:43.999652 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 22:01:43.999740 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:43.999721 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls podName:d7e6f82c-4cc2-4610-892d-9e2581c1264a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:44.999704679 +0000 UTC m=+35.121403439 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls") pod "dns-default-hmlp2" (UID: "d7e6f82c-4cc2-4610-892d-9e2581c1264a") : secret "dns-default-metrics-tls" not found Sep 10 22:01:44.100563 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:44.100519 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret\") pod \"global-pull-secret-syncer-6dx6d\" (UID: \"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7\") " pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:44.100783 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:44.100646 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs\") pod \"network-metrics-daemon-lvd5t\" (UID: \"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0\") " pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:44.100783 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:44.100770 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:01:44.100924 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:44.100856 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs podName:6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0 nodeName:}" failed. No retries permitted until 2026-09-10 22:02:16.100834348 +0000 UTC m=+66.222533108 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs") pod "network-metrics-daemon-lvd5t" (UID: "6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:01:44.103495 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:44.103465 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7-original-pull-secret\") pod \"global-pull-secret-syncer-6dx6d\" (UID: \"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7\") " pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:44.201182 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:44.201087 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtw57\" (UniqueName: \"kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57\") pod \"network-check-target-kfbmj\" (UID: \"57428f1e-0c76-4457-937b-b33b503cb09a\") " pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:44.201364 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:44.201317 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 22:01:44.201364 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:44.201346 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 22:01:44.201364 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:44.201360 2576 projected.go:196] Error preparing data for projected volume kube-api-access-gtw57 for pod openshift-network-diagnostics/network-check-target-kfbmj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:01:44.201520 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:44.201432 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57 podName:57428f1e-0c76-4457-937b-b33b503cb09a nodeName:}" failed. No retries permitted until 2026-09-10 22:02:16.201411701 +0000 UTC m=+66.323110476 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gtw57" (UniqueName: "kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57") pod "network-check-target-kfbmj" (UID: "57428f1e-0c76-4457-937b-b33b503cb09a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:01:44.336127 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:44.336086 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dx6d" Sep 10 22:01:44.425593 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:44.425555 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:01:44.425831 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:44.425555 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:01:44.428355 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:44.428333 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 22:01:44.428867 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:44.428839 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vfb6l\"" Sep 10 22:01:44.429321 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:44.428991 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 22:01:44.429321 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:44.429090 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 22:01:44.429321 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:44.429090 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5csvl\"" Sep 10 22:01:44.905970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:44.905757 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:44.906182 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:44.905926 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 22:01:44.906182 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:44.906044 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cdb65cf65-bnlmv: secret "image-registry-tls" not found Sep 10 22:01:44.906182 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:44.906100 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls podName:31f47884-061b-4eb7-b27b-91d855d9bf3a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:46.906085496 +0000 UTC m=+37.027784256 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls") pod "image-registry-6cdb65cf65-bnlmv" (UID: "31f47884-061b-4eb7-b27b-91d855d9bf3a") : secret "image-registry-tls" not found Sep 10 22:01:44.953038 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:44.952896 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6dx6d"] Sep 10 22:01:44.960984 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:44.960670 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7tzf6"] Sep 10 22:01:44.975736 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:44.975702 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod291b13a2_ed89_49f1_98e8_9c9bd3b7f7a7.slice/crio-d6b07aa93f07f39d29654f39c8e4fe100112a675b1f506b6c6f9ade9514920c2 WatchSource:0}: Error finding container d6b07aa93f07f39d29654f39c8e4fe100112a675b1f506b6c6f9ade9514920c2: Status 404 returned error can't find the container with id d6b07aa93f07f39d29654f39c8e4fe100112a675b1f506b6c6f9ade9514920c2 Sep 10 22:01:44.978133 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:01:44.978104 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9294b995_aea4_496b_8e2c_9739ceda5668.slice/crio-66b62e470aaba1ba7fb145143b5bf10c0ed188ff6f017fa5b7cb151fcaf39da7 WatchSource:0}: Error finding container 66b62e470aaba1ba7fb145143b5bf10c0ed188ff6f017fa5b7cb151fcaf39da7: Status 404 returned error can't find the container with id 66b62e470aaba1ba7fb145143b5bf10c0ed188ff6f017fa5b7cb151fcaf39da7 Sep 10 22:01:45.006950 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:45.006926 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:45.007084 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:45.007066 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 22:01:45.007138 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:45.007129 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls podName:d7e6f82c-4cc2-4610-892d-9e2581c1264a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:47.007109797 +0000 UTC m=+37.128808556 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls") pod "dns-default-hmlp2" (UID: "d7e6f82c-4cc2-4610-892d-9e2581c1264a") : secret "dns-default-metrics-tls" not found Sep 10 22:01:45.689318 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:45.689229 2576 generic.go:356] "Generic (PLEG): container finished" podID="729bf7d2-7285-450d-8a22-f13dd26be9d8" containerID="9bd4359a66764adf9a0d7520eadc6bb829e65c66d59856f38f541ea57c5891d4" exitCode=0 Sep 10 22:01:45.690065 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:45.689314 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjbjk" event={"ID":"729bf7d2-7285-450d-8a22-f13dd26be9d8","Type":"ContainerDied","Data":"9bd4359a66764adf9a0d7520eadc6bb829e65c66d59856f38f541ea57c5891d4"} Sep 10 22:01:45.690834 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:45.690764 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6dx6d" event={"ID":"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7","Type":"ContainerStarted","Data":"d6b07aa93f07f39d29654f39c8e4fe100112a675b1f506b6c6f9ade9514920c2"} Sep 10 22:01:45.692156 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:45.692126 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7tzf6" event={"ID":"9294b995-aea4-496b-8e2c-9739ceda5668","Type":"ContainerStarted","Data":"66b62e470aaba1ba7fb145143b5bf10c0ed188ff6f017fa5b7cb151fcaf39da7"} Sep 10 22:01:46.698728 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:46.698689 2576 generic.go:356] "Generic (PLEG): container finished" podID="729bf7d2-7285-450d-8a22-f13dd26be9d8" containerID="7802f90b4aaa4354b5ab41f239bd55b0b95397710acf883dc69776b3c331b25a" exitCode=0 Sep 10 22:01:46.699404 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:46.698747 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjbjk" event={"ID":"729bf7d2-7285-450d-8a22-f13dd26be9d8","Type":"ContainerDied","Data":"7802f90b4aaa4354b5ab41f239bd55b0b95397710acf883dc69776b3c331b25a"} Sep 10 22:01:46.898376 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:46.898339 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:46.923013 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:46.922941 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:46.923269 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:46.923063 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 22:01:46.923269 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:46.923079 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cdb65cf65-bnlmv: secret "image-registry-tls" not found Sep 10 22:01:46.923269 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:46.923136 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls podName:31f47884-061b-4eb7-b27b-91d855d9bf3a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:50.923117433 +0000 UTC m=+41.044816199 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls") pod "image-registry-6cdb65cf65-bnlmv" (UID: "31f47884-061b-4eb7-b27b-91d855d9bf3a") : secret "image-registry-tls" not found Sep 10 22:01:46.999781 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:46.999691 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:47.024390 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:47.024007 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:47.024390 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:47.024193 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 22:01:47.024390 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:47.024292 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls podName:d7e6f82c-4cc2-4610-892d-9e2581c1264a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:51.024270347 +0000 UTC m=+41.145969113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls") pod "dns-default-hmlp2" (UID: "d7e6f82c-4cc2-4610-892d-9e2581c1264a") : secret "dns-default-metrics-tls" not found Sep 10 22:01:48.684716 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:48.684466 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:48.708635 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:48.708598 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjbjk" event={"ID":"729bf7d2-7285-450d-8a22-f13dd26be9d8","Type":"ContainerStarted","Data":"6775100d758f2e203041aa17a937eae8b3e7ab09d2d7f6a85ca575addad39526"} Sep 10 22:01:48.729982 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:48.729927 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pjbjk" podStartSLOduration=8.673191124 podStartE2EDuration="38.729906218s" totalImagesPullingTime="30.056715094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:01:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:01:48.728661355 +0000 UTC m=+38.850360138" watchObservedRunningTime="2026-09-10 22:01:48.729906218 +0000 UTC m=+38.851605001" Sep 10 22:01:49.712486 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:49.712402 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7tzf6" event={"ID":"9294b995-aea4-496b-8e2c-9739ceda5668","Type":"ContainerStarted","Data":"90ac1de95e3e5b5ab1a0ceaf7802a9a916bd9727396e15ef82d1a76018a1cd30"} Sep 10 22:01:49.728229 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:49.728174 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7tzf6" podStartSLOduration=2.297734015 podStartE2EDuration="6.728156721s" totalImagesPullingTime="4.430422706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:01:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:01:49.726763569 +0000 UTC m=+39.848462342" watchObservedRunningTime="2026-09-10 22:01:49.728156721 +0000 UTC m=+39.849855503" Sep 10 22:01:50.716520 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:50.716480 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6dx6d" event={"ID":"291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7","Type":"ContainerStarted","Data":"7753f47744b52339f0484150e9458879ed8da8de64a70c36640c11ba91913228"} Sep 10 22:01:50.731182 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:50.731129 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6dx6d" podStartSLOduration=34.934079064 podStartE2EDuration="39.731111216s" totalImagesPullingTime="4.797032152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:01:50.730520333 +0000 UTC m=+40.852219114" watchObservedRunningTime="2026-09-10 22:01:50.731111216 +0000 UTC m=+40.852809997" Sep 10 22:01:50.926948 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:50.926907 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:50.957008 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:50.956968 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:50.957138 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:50.957115 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 22:01:50.957138 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:50.957134 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cdb65cf65-bnlmv: secret "image-registry-tls" not found Sep 10 22:01:50.957227 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:50.957190 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls podName:31f47884-061b-4eb7-b27b-91d855d9bf3a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:58.957174367 +0000 UTC m=+49.078873127 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls") pod "image-registry-6cdb65cf65-bnlmv" (UID: "31f47884-061b-4eb7-b27b-91d855d9bf3a") : secret "image-registry-tls" not found Sep 10 22:01:51.057700 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:51.057616 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:51.057875 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:51.057724 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 22:01:51.057875 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:51.057773 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls podName:d7e6f82c-4cc2-4610-892d-9e2581c1264a nodeName:}" failed. No retries permitted until 2026-09-10 22:01:59.057760023 +0000 UTC m=+49.179458784 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls") pod "dns-default-hmlp2" (UID: "d7e6f82c-4cc2-4610-892d-9e2581c1264a") : secret "dns-default-metrics-tls" not found Sep 10 22:01:53.046425 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:53.046389 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:57.120627 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:57.120581 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:57.218526 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:57.218498 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:57.948535 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:57.948504 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:58.387144 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:58.387098 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:58.446049 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:58.446012 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:58.509384 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:58.509356 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:58.591611 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:58.591582 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:58.648635 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:58.648565 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:59.012055 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:59.011960 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:01:59.012205 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:59.012123 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 22:01:59.012205 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:59.012137 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cdb65cf65-bnlmv: secret "image-registry-tls" not found Sep 10 22:01:59.012205 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:59.012196 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls podName:31f47884-061b-4eb7-b27b-91d855d9bf3a nodeName:}" failed. No retries permitted until 2026-09-10 22:02:15.012178582 +0000 UTC m=+65.133877346 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls") pod "image-registry-6cdb65cf65-bnlmv" (UID: "31f47884-061b-4eb7-b27b-91d855d9bf3a") : secret "image-registry-tls" not found Sep 10 22:01:59.029484 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:59.029445 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:01:59.113207 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:01:59.113175 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:01:59.113354 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:59.113313 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 22:01:59.113391 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:01:59.113370 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls podName:d7e6f82c-4cc2-4610-892d-9e2581c1264a nodeName:}" failed. No retries permitted until 2026-09-10 22:02:15.113354197 +0000 UTC m=+65.235052958 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls") pod "dns-default-hmlp2" (UID: "d7e6f82c-4cc2-4610-892d-9e2581c1264a") : secret "dns-default-metrics-tls" not found Sep 10 22:02:09.288357 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:09.288328 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:09.341447 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:09.341408 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:09.680974 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:09.680949 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sk86l" Sep 10 22:02:09.826518 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:09.826488 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:09.881426 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:09.881397 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:12.165422 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:12.165387 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:12.220948 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:12.220919 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:14.436742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:14.436704 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:14.537672 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:14.537641 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:15.021337 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:15.021303 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:02:15.021518 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:15.021448 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 22:02:15.021518 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:15.021461 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cdb65cf65-bnlmv: secret "image-registry-tls" not found Sep 10 22:02:15.021592 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:15.021520 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls podName:31f47884-061b-4eb7-b27b-91d855d9bf3a nodeName:}" failed. No retries permitted until 2026-09-10 22:02:47.0215034 +0000 UTC m=+97.143202164 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls") pod "image-registry-6cdb65cf65-bnlmv" (UID: "31f47884-061b-4eb7-b27b-91d855d9bf3a") : secret "image-registry-tls" not found Sep 10 22:02:15.122122 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:15.122086 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:02:15.122261 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:15.122215 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 22:02:15.122299 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:15.122265 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls podName:d7e6f82c-4cc2-4610-892d-9e2581c1264a nodeName:}" failed. No retries permitted until 2026-09-10 22:02:47.122252027 +0000 UTC m=+97.243950787 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls") pod "dns-default-hmlp2" (UID: "d7e6f82c-4cc2-4610-892d-9e2581c1264a") : secret "dns-default-metrics-tls" not found Sep 10 22:02:16.129442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:16.129401 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs\") pod \"network-metrics-daemon-lvd5t\" (UID: \"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0\") " pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:02:16.131782 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:16.131760 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 22:02:16.140426 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:16.140395 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 22:02:16.140601 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:16.140502 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs podName:6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0 nodeName:}" failed. No retries permitted until 2026-09-10 22:03:20.140478116 +0000 UTC m=+130.262176882 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs") pod "network-metrics-daemon-lvd5t" (UID: "6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0") : secret "metrics-daemon-secret" not found Sep 10 22:02:16.230118 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:16.230077 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtw57\" (UniqueName: \"kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57\") pod \"network-check-target-kfbmj\" (UID: \"57428f1e-0c76-4457-937b-b33b503cb09a\") " pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:02:16.232098 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:16.232078 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 22:02:16.242853 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:16.242826 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 22:02:16.255387 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:16.255360 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtw57\" (UniqueName: \"kubernetes.io/projected/57428f1e-0c76-4457-937b-b33b503cb09a-kube-api-access-gtw57\") pod \"network-check-target-kfbmj\" (UID: \"57428f1e-0c76-4457-937b-b33b503cb09a\") " pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:02:16.540200 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:16.540120 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5csvl\"" Sep 10 22:02:16.548958 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:16.548928 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:02:16.819567 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:16.819537 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kfbmj"] Sep 10 22:02:16.822955 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:02:16.822925 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57428f1e_0c76_4457_937b_b33b503cb09a.slice/crio-7a5fd5646f56c1a4fb962b22715f88acc5b6b9f03f465235292f6bc3245b1cf6 WatchSource:0}: Error finding container 7a5fd5646f56c1a4fb962b22715f88acc5b6b9f03f465235292f6bc3245b1cf6: Status 404 returned error can't find the container with id 7a5fd5646f56c1a4fb962b22715f88acc5b6b9f03f465235292f6bc3245b1cf6 Sep 10 22:02:17.784239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:17.784197 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kfbmj" event={"ID":"57428f1e-0c76-4457-937b-b33b503cb09a","Type":"ContainerStarted","Data":"7a5fd5646f56c1a4fb962b22715f88acc5b6b9f03f465235292f6bc3245b1cf6"} Sep 10 22:02:18.923972 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:18.923930 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:18.990330 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:18.990279 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:20.792566 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:20.792533 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kfbmj" event={"ID":"57428f1e-0c76-4457-937b-b33b503cb09a","Type":"ContainerStarted","Data":"53c8585636ddf724b195caff89cbd5d4df799c03a4eba5afc939102971e22c69"} Sep 10 22:02:20.792961 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:20.792665 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:02:20.807198 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:20.807144 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kfbmj" podStartSLOduration=67.88637303 podStartE2EDuration="1m10.807125164s" totalImagesPullingTime="2.920752134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:02:20.806255445 +0000 UTC m=+70.927954238" watchObservedRunningTime="2026-09-10 22:02:20.807125164 +0000 UTC m=+70.928823946" Sep 10 22:02:28.465071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.465033 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-ktm6m"] Sep 10 22:02:28.467607 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.467590 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr"] Sep 10 22:02:28.467724 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.467705 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-ktm6m" Sep 10 22:02:28.469556 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.469535 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:02:28.469795 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.469774 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-ljrkv\"" Sep 10 22:02:28.469920 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.469902 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 10 22:02:28.470015 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.469998 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 10 22:02:28.471420 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.471336 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 10 22:02:28.472061 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.472046 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 22:02:28.472263 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.472243 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 22:02:28.472263 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.472262 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gq27s\"" Sep 10 22:02:28.472469 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.472377 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 10 22:02:28.479240 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.479217 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-ktm6m"] Sep 10 22:02:28.480366 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.480342 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr"] Sep 10 22:02:28.517706 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.517667 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mv5r\" (UniqueName: \"kubernetes.io/projected/39a311e1-4d1f-4d6e-b3de-752d6e620252-kube-api-access-6mv5r\") pod \"volume-data-source-validator-86695b7cc7-ktm6m\" (UID: \"39a311e1-4d1f-4d6e-b3de-752d6e620252\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-ktm6m" Sep 10 22:02:28.517936 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.517751 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/efc199a6-0151-4410-a431-d8652e0541f4-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-qbvrr\" (UID: \"efc199a6-0151-4410-a431-d8652e0541f4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:02:28.517936 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.517780 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lqwq\" (UniqueName: \"kubernetes.io/projected/efc199a6-0151-4410-a431-d8652e0541f4-kube-api-access-5lqwq\") pod \"cluster-monitoring-operator-6cd954b897-qbvrr\" (UID: \"efc199a6-0151-4410-a431-d8652e0541f4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:02:28.517936 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.517837 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-qbvrr\" (UID: \"efc199a6-0151-4410-a431-d8652e0541f4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:02:28.556223 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.556185 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5"] Sep 10 22:02:28.558184 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.558158 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" Sep 10 22:02:28.562610 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.562585 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 10 22:02:28.562929 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.562912 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 10 22:02:28.563499 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.563476 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-bsqcv\"" Sep 10 22:02:28.563603 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.563527 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 10 22:02:28.563603 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.563530 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 10 22:02:28.579613 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.579582 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5"] Sep 10 22:02:28.618489 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.618451 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1566f5ee-e336-455a-b28c-a77e91708be2-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-2t7c5\" (UID: \"1566f5ee-e336-455a-b28c-a77e91708be2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" Sep 10 22:02:28.618489 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.618487 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1566f5ee-e336-455a-b28c-a77e91708be2-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-2t7c5\" (UID: \"1566f5ee-e336-455a-b28c-a77e91708be2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" Sep 10 22:02:28.618689 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.618510 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/efc199a6-0151-4410-a431-d8652e0541f4-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-qbvrr\" (UID: \"efc199a6-0151-4410-a431-d8652e0541f4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:02:28.618689 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.618529 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lqwq\" (UniqueName: \"kubernetes.io/projected/efc199a6-0151-4410-a431-d8652e0541f4-kube-api-access-5lqwq\") pod \"cluster-monitoring-operator-6cd954b897-qbvrr\" (UID: \"efc199a6-0151-4410-a431-d8652e0541f4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:02:28.618689 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.618556 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkw89\" (UniqueName: \"kubernetes.io/projected/1566f5ee-e336-455a-b28c-a77e91708be2-kube-api-access-fkw89\") pod \"kube-storage-version-migrator-operator-6c8f84f897-2t7c5\" (UID: \"1566f5ee-e336-455a-b28c-a77e91708be2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" Sep 10 22:02:28.618689 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.618578 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-qbvrr\" (UID: \"efc199a6-0151-4410-a431-d8652e0541f4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:02:28.618689 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.618632 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mv5r\" (UniqueName: \"kubernetes.io/projected/39a311e1-4d1f-4d6e-b3de-752d6e620252-kube-api-access-6mv5r\") pod \"volume-data-source-validator-86695b7cc7-ktm6m\" (UID: \"39a311e1-4d1f-4d6e-b3de-752d6e620252\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-ktm6m" Sep 10 22:02:28.618876 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:28.618711 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 22:02:28.618876 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:28.618788 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls podName:efc199a6-0151-4410-a431-d8652e0541f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:02:29.118767713 +0000 UTC m=+79.240466486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-qbvrr" (UID: "efc199a6-0151-4410-a431-d8652e0541f4") : secret "cluster-monitoring-operator-tls" not found Sep 10 22:02:28.619335 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.619317 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/efc199a6-0151-4410-a431-d8652e0541f4-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-qbvrr\" (UID: \"efc199a6-0151-4410-a431-d8652e0541f4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:02:28.626773 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.626746 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lqwq\" (UniqueName: \"kubernetes.io/projected/efc199a6-0151-4410-a431-d8652e0541f4-kube-api-access-5lqwq\") pod \"cluster-monitoring-operator-6cd954b897-qbvrr\" (UID: \"efc199a6-0151-4410-a431-d8652e0541f4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:02:28.627012 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.626989 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mv5r\" (UniqueName: \"kubernetes.io/projected/39a311e1-4d1f-4d6e-b3de-752d6e620252-kube-api-access-6mv5r\") pod \"volume-data-source-validator-86695b7cc7-ktm6m\" (UID: \"39a311e1-4d1f-4d6e-b3de-752d6e620252\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-ktm6m" Sep 10 22:02:28.719016 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.718911 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1566f5ee-e336-455a-b28c-a77e91708be2-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-2t7c5\" (UID: \"1566f5ee-e336-455a-b28c-a77e91708be2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" Sep 10 22:02:28.719016 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.718954 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1566f5ee-e336-455a-b28c-a77e91708be2-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-2t7c5\" (UID: \"1566f5ee-e336-455a-b28c-a77e91708be2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" Sep 10 22:02:28.719016 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.718976 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkw89\" (UniqueName: \"kubernetes.io/projected/1566f5ee-e336-455a-b28c-a77e91708be2-kube-api-access-fkw89\") pod \"kube-storage-version-migrator-operator-6c8f84f897-2t7c5\" (UID: \"1566f5ee-e336-455a-b28c-a77e91708be2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" Sep 10 22:02:28.719544 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.719522 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1566f5ee-e336-455a-b28c-a77e91708be2-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-2t7c5\" (UID: \"1566f5ee-e336-455a-b28c-a77e91708be2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" Sep 10 22:02:28.721269 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.721246 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1566f5ee-e336-455a-b28c-a77e91708be2-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-2t7c5\" (UID: \"1566f5ee-e336-455a-b28c-a77e91708be2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" Sep 10 22:02:28.730599 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.730564 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkw89\" (UniqueName: \"kubernetes.io/projected/1566f5ee-e336-455a-b28c-a77e91708be2-kube-api-access-fkw89\") pod \"kube-storage-version-migrator-operator-6c8f84f897-2t7c5\" (UID: \"1566f5ee-e336-455a-b28c-a77e91708be2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" Sep 10 22:02:28.777295 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.777254 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-ktm6m" Sep 10 22:02:28.787725 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.787697 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:28.867612 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.867554 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" Sep 10 22:02:28.888322 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:28.888285 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:29.043965 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:29.043885 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-ktm6m"] Sep 10 22:02:29.047227 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:02:29.047203 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a311e1_4d1f_4d6e_b3de_752d6e620252.slice/crio-841f14b0c30b62801ca990003cd2fc598a9353a6c1f19883941085f24e820737 WatchSource:0}: Error finding container 841f14b0c30b62801ca990003cd2fc598a9353a6c1f19883941085f24e820737: Status 404 returned error can't find the container with id 841f14b0c30b62801ca990003cd2fc598a9353a6c1f19883941085f24e820737 Sep 10 22:02:29.123361 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:29.123326 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-qbvrr\" (UID: \"efc199a6-0151-4410-a431-d8652e0541f4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:02:29.123549 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:29.123469 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 22:02:29.123611 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:29.123560 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls podName:efc199a6-0151-4410-a431-d8652e0541f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:02:30.123538549 +0000 UTC m=+80.245237314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-qbvrr" (UID: "efc199a6-0151-4410-a431-d8652e0541f4") : secret "cluster-monitoring-operator-tls" not found Sep 10 22:02:29.137241 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:29.137210 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5"] Sep 10 22:02:29.140164 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:02:29.140132 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1566f5ee_e336_455a_b28c_a77e91708be2.slice/crio-b2473f1e5e91ba78720d70806f70eed530248a11b5ff9e903b6b46811e071116 WatchSource:0}: Error finding container b2473f1e5e91ba78720d70806f70eed530248a11b5ff9e903b6b46811e071116: Status 404 returned error can't find the container with id b2473f1e5e91ba78720d70806f70eed530248a11b5ff9e903b6b46811e071116 Sep 10 22:02:29.818050 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:29.818000 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-ktm6m" event={"ID":"39a311e1-4d1f-4d6e-b3de-752d6e620252","Type":"ContainerStarted","Data":"841f14b0c30b62801ca990003cd2fc598a9353a6c1f19883941085f24e820737"} Sep 10 22:02:29.819778 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:29.819740 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" event={"ID":"1566f5ee-e336-455a-b28c-a77e91708be2","Type":"ContainerStarted","Data":"b2473f1e5e91ba78720d70806f70eed530248a11b5ff9e903b6b46811e071116"} Sep 10 22:02:30.132776 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:30.132675 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-qbvrr\" (UID: \"efc199a6-0151-4410-a431-d8652e0541f4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:02:30.132965 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:30.132878 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 22:02:30.133026 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:30.132971 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls podName:efc199a6-0151-4410-a431-d8652e0541f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:02:32.132948886 +0000 UTC m=+82.254647667 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-qbvrr" (UID: "efc199a6-0151-4410-a431-d8652e0541f4") : secret "cluster-monitoring-operator-tls" not found Sep 10 22:02:30.824425 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:30.824387 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-ktm6m" event={"ID":"39a311e1-4d1f-4d6e-b3de-752d6e620252","Type":"ContainerStarted","Data":"94c267aa8161efb25d6956d20fe8b5c77d68d25e7a6b3a46c50aefea2993fb7e"} Sep 10 22:02:30.838529 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:30.838472 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-ktm6m" podStartSLOduration=1.688151376 podStartE2EDuration="2.838455753s" totalImagesPullingTime="1.150304377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:02:30.837480283 +0000 UTC m=+80.959179065" watchObservedRunningTime="2026-09-10 22:02:30.838455753 +0000 UTC m=+80.960154535" Sep 10 22:02:30.947260 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:30.947226 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:31.003627 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:31.003590 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:31.828981 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:31.828940 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" event={"ID":"1566f5ee-e336-455a-b28c-a77e91708be2","Type":"ContainerStarted","Data":"44da26366d214d738fbc9cf04413652aafd5bf361ad86de423ec25584d9fd625"} Sep 10 22:02:31.846252 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:31.846190 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" podStartSLOduration=1.8537730510000001 podStartE2EDuration="3.846168268s" totalImagesPullingTime="1.992395217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:02:31.845105103 +0000 UTC m=+81.966803886" watchObservedRunningTime="2026-09-10 22:02:31.846168268 +0000 UTC m=+81.967867050" Sep 10 22:02:32.147823 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:32.147712 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-qbvrr\" (UID: \"efc199a6-0151-4410-a431-d8652e0541f4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:02:32.147967 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:32.147840 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 22:02:32.147967 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:32.147895 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls podName:efc199a6-0151-4410-a431-d8652e0541f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:02:36.147881614 +0000 UTC m=+86.269580374 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-qbvrr" (UID: "efc199a6-0151-4410-a431-d8652e0541f4") : secret "cluster-monitoring-operator-tls" not found Sep 10 22:02:32.285009 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:32.284902 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:32.384085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:32.384054 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:32.486062 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:32.485965 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:33.142355 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:33.142326 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:33.245606 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:33.245562 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:33.543462 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:33.543373 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:33.824989 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:33.824960 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:33.931068 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:33.931039 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:35.026931 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.026896 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gpnrt"] Sep 10 22:02:35.029233 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.029214 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.031195 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.031174 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fvnxt\"" Sep 10 22:02:35.031295 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.031179 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 22:02:35.031735 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.031719 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 22:02:35.031790 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.031756 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 22:02:35.031860 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.031821 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 22:02:35.047546 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.047517 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:35.047876 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.047855 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gpnrt"] Sep 10 22:02:35.070848 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.070814 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5dd313b9-389a-4562-a835-824681d8e89e-crio-socket\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.070848 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.070851 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4fmz\" (UniqueName: \"kubernetes.io/projected/5dd313b9-389a-4562-a835-824681d8e89e-kube-api-access-j4fmz\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.071116 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.070872 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.071116 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.070942 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5dd313b9-389a-4562-a835-824681d8e89e-data-volume\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.071116 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.070959 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.114315 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.114279 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:35.172135 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.172098 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5dd313b9-389a-4562-a835-824681d8e89e-crio-socket\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.172135 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.172137 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4fmz\" (UniqueName: \"kubernetes.io/projected/5dd313b9-389a-4562-a835-824681d8e89e-kube-api-access-j4fmz\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.172323 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.172155 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.172323 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.172207 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5dd313b9-389a-4562-a835-824681d8e89e-data-volume\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.172323 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.172223 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5dd313b9-389a-4562-a835-824681d8e89e-crio-socket\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.172323 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.172223 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.172323 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:35.172296 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 22:02:35.172534 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:35.172321 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 22:02:35.172534 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:35.172347 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls podName:5dd313b9-389a-4562-a835-824681d8e89e nodeName:}" failed. No retries permitted until 2026-09-10 22:02:35.672333757 +0000 UTC m=+85.794032517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gpnrt" (UID: "5dd313b9-389a-4562-a835-824681d8e89e") : secret "insights-runtime-extractor-tls" not found Sep 10 22:02:35.172534 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:35.172388 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm podName:5dd313b9-389a-4562-a835-824681d8e89e nodeName:}" failed. No retries permitted until 2026-09-10 22:02:35.672370075 +0000 UTC m=+85.794068850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gpnrt" (UID: "5dd313b9-389a-4562-a835-824681d8e89e") : configmap "kube-rbac-proxy" not found Sep 10 22:02:35.172639 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.172546 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5dd313b9-389a-4562-a835-824681d8e89e-data-volume\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.186244 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.186211 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4fmz\" (UniqueName: \"kubernetes.io/projected/5dd313b9-389a-4562-a835-824681d8e89e-kube-api-access-j4fmz\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.315209 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.315147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:35.416356 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.416323 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:35.485398 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.485362 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:35.676249 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.676168 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.676249 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:35.676244 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:35.676422 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:35.676297 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 22:02:35.676422 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:35.676356 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 22:02:35.676422 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:35.676404 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm podName:5dd313b9-389a-4562-a835-824681d8e89e nodeName:}" failed. No retries permitted until 2026-09-10 22:02:36.676388288 +0000 UTC m=+86.798087048 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gpnrt" (UID: "5dd313b9-389a-4562-a835-824681d8e89e") : configmap "kube-rbac-proxy" not found Sep 10 22:02:35.676422 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:35.676420 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls podName:5dd313b9-389a-4562-a835-824681d8e89e nodeName:}" failed. No retries permitted until 2026-09-10 22:02:36.676413913 +0000 UTC m=+86.798112672 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gpnrt" (UID: "5dd313b9-389a-4562-a835-824681d8e89e") : secret "insights-runtime-extractor-tls" not found Sep 10 22:02:36.180273 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:36.180235 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-qbvrr\" (UID: \"efc199a6-0151-4410-a431-d8652e0541f4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:02:36.180633 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:36.180378 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 22:02:36.180633 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:36.180448 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls podName:efc199a6-0151-4410-a431-d8652e0541f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:02:44.180431966 +0000 UTC m=+94.302130760 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-qbvrr" (UID: "efc199a6-0151-4410-a431-d8652e0541f4") : secret "cluster-monitoring-operator-tls" not found Sep 10 22:02:36.435601 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:36.435522 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:36.507086 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:36.507055 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:36.685087 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:36.685051 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:36.685255 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:36.685120 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:36.685255 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:36.685210 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 22:02:36.685255 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:36.685243 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 22:02:36.685379 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:36.685275 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls podName:5dd313b9-389a-4562-a835-824681d8e89e nodeName:}" failed. No retries permitted until 2026-09-10 22:02:38.685260321 +0000 UTC m=+88.806959086 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gpnrt" (UID: "5dd313b9-389a-4562-a835-824681d8e89e") : secret "insights-runtime-extractor-tls" not found Sep 10 22:02:36.685379 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:36.685298 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm podName:5dd313b9-389a-4562-a835-824681d8e89e nodeName:}" failed. No retries permitted until 2026-09-10 22:02:38.685284553 +0000 UTC m=+88.806983329 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gpnrt" (UID: "5dd313b9-389a-4562-a835-824681d8e89e") : configmap "kube-rbac-proxy" not found Sep 10 22:02:36.703356 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:36.703281 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:36.800996 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:36.800966 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:37.600408 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:37.600380 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8z8zh_bd1bec7b-f106-4ca2-b72d-71a7fe7a1929/dns-node-resolver/0.log" Sep 10 22:02:38.118707 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:38.118680 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:38.219499 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:38.219456 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:38.334434 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:38.334407 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:38.426507 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:38.426418 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:38.700213 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:38.700119 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:38.700213 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:38.700175 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:38.700662 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:38.700245 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 22:02:38.700662 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:38.700300 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm podName:5dd313b9-389a-4562-a835-824681d8e89e nodeName:}" failed. No retries permitted until 2026-09-10 22:02:42.700286337 +0000 UTC m=+92.821985097 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gpnrt" (UID: "5dd313b9-389a-4562-a835-824681d8e89e") : configmap "kube-rbac-proxy" not found Sep 10 22:02:38.700662 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:38.700317 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 22:02:38.700662 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:38.700409 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls podName:5dd313b9-389a-4562-a835-824681d8e89e nodeName:}" failed. No retries permitted until 2026-09-10 22:02:42.700389252 +0000 UTC m=+92.822088029 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gpnrt" (UID: "5dd313b9-389a-4562-a835-824681d8e89e") : secret "insights-runtime-extractor-tls" not found Sep 10 22:02:38.807447 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:38.807413 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:39.002286 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:39.002209 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zfmq5_a95cad0a-1c31-4ecf-915e-05b752bdaeee/node-ca/0.log" Sep 10 22:02:39.131452 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:39.131425 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:39.823434 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:39.823406 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7tzf6_9294b995-aea4-496b-8e2c-9739ceda5668/serve-healthcheck-canary/0.log" Sep 10 22:02:39.881456 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:39.881425 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:40.215831 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:40.215744 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:40.401322 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:40.401295 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-2t7c5_1566f5ee-e336-455a-b28c-a77e91708be2/kube-storage-version-migrator-operator/0.log" Sep 10 22:02:40.425776 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:40.425745 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:40.604846 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:40.604787 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:41.082554 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:41.082525 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:41.602686 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:41.602655 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:41.829204 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:41.829172 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:41.997461 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:41.997384 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:42.311983 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:42.311906 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:42.412765 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:42.412733 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:42.626140 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:42.626109 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:42.697109 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:42.697078 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:42.734337 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:42.734303 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:42.734454 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:42.734436 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 22:02:42.734501 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:42.734460 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:42.734541 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:42.734506 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm podName:5dd313b9-389a-4562-a835-824681d8e89e nodeName:}" failed. No retries permitted until 2026-09-10 22:02:50.734491318 +0000 UTC m=+100.856190098 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gpnrt" (UID: "5dd313b9-389a-4562-a835-824681d8e89e") : configmap "kube-rbac-proxy" not found Sep 10 22:02:42.734581 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:42.734563 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 22:02:42.734632 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:42.734622 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls podName:5dd313b9-389a-4562-a835-824681d8e89e nodeName:}" failed. No retries permitted until 2026-09-10 22:02:50.734605994 +0000 UTC m=+100.856304755 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gpnrt" (UID: "5dd313b9-389a-4562-a835-824681d8e89e") : secret "insights-runtime-extractor-tls" not found Sep 10 22:02:42.997480 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:42.997396 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:43.053835 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:43.053772 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:43.610900 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:43.610870 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:44.009725 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:44.009627 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:44.214153 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:44.214123 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:44.248257 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:44.248226 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-qbvrr\" (UID: \"efc199a6-0151-4410-a431-d8652e0541f4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:02:44.248395 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:44.248378 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 22:02:44.248454 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:02:44.248445 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls podName:efc199a6-0151-4410-a431-d8652e0541f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:03:00.248427438 +0000 UTC m=+110.370126210 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-qbvrr" (UID: "efc199a6-0151-4410-a431-d8652e0541f4") : secret "cluster-monitoring-operator-tls" not found Sep 10 22:02:44.316168 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:44.316135 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:44.606338 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:44.606256 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:45.204422 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:45.204386 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:45.481106 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:45.481026 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:45.806228 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:45.806142 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:46.004054 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:46.004019 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:46.204610 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:46.204580 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:46.620537 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:46.620511 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:46.904641 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:46.904563 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:46.959479 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:46.959447 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:47.032630 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.032602 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:47.066462 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.066428 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:02:47.068785 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.068762 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls\") pod \"image-registry-6cdb65cf65-bnlmv\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:02:47.141547 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.141516 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c5lwm\"" Sep 10 22:02:47.149571 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.149544 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:02:47.167009 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.166983 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:02:47.169736 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.169711 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7e6f82c-4cc2-4610-892d-9e2581c1264a-metrics-tls\") pod \"dns-default-hmlp2\" (UID: \"d7e6f82c-4cc2-4610-892d-9e2581c1264a\") " pod="openshift-dns/dns-default-hmlp2" Sep 10 22:02:47.425130 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.425047 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cdb65cf65-bnlmv"] Sep 10 22:02:47.428606 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:02:47.428568 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31f47884_061b_4eb7_b27b_91d855d9bf3a.slice/crio-80ab84a2c2e3598507f8e4a3b7c2b75d38542a2345ad4ed8e6d6d633443e03c7 WatchSource:0}: Error finding container 80ab84a2c2e3598507f8e4a3b7c2b75d38542a2345ad4ed8e6d6d633443e03c7: Status 404 returned error can't find the container with id 80ab84a2c2e3598507f8e4a3b7c2b75d38542a2345ad4ed8e6d6d633443e03c7 Sep 10 22:02:47.454449 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.454417 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qt6wb\"" Sep 10 22:02:47.462414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.462384 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hmlp2" Sep 10 22:02:47.633982 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.633945 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:47.738940 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.738909 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hmlp2"] Sep 10 22:02:47.742219 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:02:47.742193 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e6f82c_4cc2_4610_892d_9e2581c1264a.slice/crio-90773f0d9620d4a2fa1e01e499b73687b8e71b15b975b325ce2a648299ad045e WatchSource:0}: Error finding container 90773f0d9620d4a2fa1e01e499b73687b8e71b15b975b325ce2a648299ad045e: Status 404 returned error can't find the container with id 90773f0d9620d4a2fa1e01e499b73687b8e71b15b975b325ce2a648299ad045e Sep 10 22:02:47.874074 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.874039 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" event={"ID":"31f47884-061b-4eb7-b27b-91d855d9bf3a","Type":"ContainerStarted","Data":"1267fd43037e4ef56559407234c946bf3b17f2730e50a965ea4b99e04951e8c8"} Sep 10 22:02:47.874246 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.874094 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" event={"ID":"31f47884-061b-4eb7-b27b-91d855d9bf3a","Type":"ContainerStarted","Data":"80ab84a2c2e3598507f8e4a3b7c2b75d38542a2345ad4ed8e6d6d633443e03c7"} Sep 10 22:02:47.874246 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.874121 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:02:47.875230 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.875203 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hmlp2" event={"ID":"d7e6f82c-4cc2-4610-892d-9e2581c1264a","Type":"ContainerStarted","Data":"90773f0d9620d4a2fa1e01e499b73687b8e71b15b975b325ce2a648299ad045e"} Sep 10 22:02:47.900059 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:47.899838 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" podStartSLOduration=96.899822234 podStartE2EDuration="1m36.899822234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:01:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:02:47.899710735 +0000 UTC m=+98.021409538" watchObservedRunningTime="2026-09-10 22:02:47.899822234 +0000 UTC m=+98.021521022" Sep 10 22:02:48.007397 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:48.007304 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:48.211305 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:48.211240 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:48.606165 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:48.606107 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:48.810648 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:48.810609 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:02:49.884352 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:49.884318 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hmlp2" event={"ID":"d7e6f82c-4cc2-4610-892d-9e2581c1264a","Type":"ContainerStarted","Data":"177e1f774ac3d0f91a7dde40c1c9dc2cd656cbd8c8c27633ae86747b6d2917de"} Sep 10 22:02:49.884719 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:49.884417 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hmlp2" event={"ID":"d7e6f82c-4cc2-4610-892d-9e2581c1264a","Type":"ContainerStarted","Data":"4293acb763e90b34a09920f8cdd2f4ca9649d2760b400756103f7b1e698b04c9"} Sep 10 22:02:49.884719 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:49.884482 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hmlp2" Sep 10 22:02:49.900568 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:49.900517 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hmlp2" podStartSLOduration=65.351459557 podStartE2EDuration="1m6.900499845s" totalImagesPullingTime="1.549040288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:01:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:02:49.899035252 +0000 UTC m=+100.020734034" watchObservedRunningTime="2026-09-10 22:02:49.900499845 +0000 UTC m=+100.022198627" Sep 10 22:02:50.795785 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:50.795741 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:50.795785 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:50.795792 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:50.796327 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:50.796306 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5dd313b9-389a-4562-a835-824681d8e89e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:50.798182 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:50.798154 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5dd313b9-389a-4562-a835-824681d8e89e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gpnrt\" (UID: \"5dd313b9-389a-4562-a835-824681d8e89e\") " pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:50.938427 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:50.938385 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gpnrt" Sep 10 22:02:51.239970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:51.239940 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gpnrt"] Sep 10 22:02:51.243470 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:02:51.243445 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dd313b9_389a_4562_a835_824681d8e89e.slice/crio-6b57e6410b437b0c36a455da0d8e7780b44a6fcd956134f1b651e4ae21064d4a WatchSource:0}: Error finding container 6b57e6410b437b0c36a455da0d8e7780b44a6fcd956134f1b651e4ae21064d4a: Status 404 returned error can't find the container with id 6b57e6410b437b0c36a455da0d8e7780b44a6fcd956134f1b651e4ae21064d4a Sep 10 22:02:51.798136 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:51.798105 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kfbmj" Sep 10 22:02:51.892720 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:51.892686 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gpnrt" event={"ID":"5dd313b9-389a-4562-a835-824681d8e89e","Type":"ContainerStarted","Data":"74c19d30abbc68534105304b14792a21f3dad6e7e83596351de10f8b4000a815"} Sep 10 22:02:51.892912 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:51.892783 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gpnrt" event={"ID":"5dd313b9-389a-4562-a835-824681d8e89e","Type":"ContainerStarted","Data":"6b57e6410b437b0c36a455da0d8e7780b44a6fcd956134f1b651e4ae21064d4a"} Sep 10 22:02:52.898562 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:52.898524 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gpnrt" event={"ID":"5dd313b9-389a-4562-a835-824681d8e89e","Type":"ContainerStarted","Data":"35d66b0c54ccb5a2fd37b6249c19af7381b03730e28a23e8c7b91b61bb11d8be"} Sep 10 22:02:53.903774 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:53.903741 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gpnrt" event={"ID":"5dd313b9-389a-4562-a835-824681d8e89e","Type":"ContainerStarted","Data":"0795ca7541e9ea374a3229d27cda864b056bdcb5c80ec1ee72d132d91016e0eb"} Sep 10 22:02:53.926049 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:53.925990 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gpnrt" podStartSLOduration=16.919087015 podStartE2EDuration="18.925973834s" totalImagesPullingTime="2.006886819s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:02:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:02:53.925159878 +0000 UTC m=+104.046858663" watchObservedRunningTime="2026-09-10 22:02:53.925973834 +0000 UTC m=+104.047672615" Sep 10 22:02:57.142918 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.142886 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd"] Sep 10 22:02:57.144859 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.144834 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" Sep 10 22:02:57.147656 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.147629 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 10 22:02:57.147794 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.147688 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 10 22:02:57.147794 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.147700 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 10 22:02:57.147794 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.147700 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 10 22:02:57.153985 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.153962 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd"] Sep 10 22:02:57.240791 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.240754 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d53242c5-034f-4b6f-bff0-b2af51ff76ac-tmp\") pod \"klusterlet-addon-workmgr-7fbccd99cb-bg2bd\" (UID: \"d53242c5-034f-4b6f-bff0-b2af51ff76ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" Sep 10 22:02:57.240791 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.240818 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d53242c5-034f-4b6f-bff0-b2af51ff76ac-klusterlet-config\") pod \"klusterlet-addon-workmgr-7fbccd99cb-bg2bd\" (UID: \"d53242c5-034f-4b6f-bff0-b2af51ff76ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" Sep 10 22:02:57.241038 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.240921 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2rhw\" (UniqueName: \"kubernetes.io/projected/d53242c5-034f-4b6f-bff0-b2af51ff76ac-kube-api-access-d2rhw\") pod \"klusterlet-addon-workmgr-7fbccd99cb-bg2bd\" (UID: \"d53242c5-034f-4b6f-bff0-b2af51ff76ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" Sep 10 22:02:57.342006 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.341968 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d53242c5-034f-4b6f-bff0-b2af51ff76ac-tmp\") pod \"klusterlet-addon-workmgr-7fbccd99cb-bg2bd\" (UID: \"d53242c5-034f-4b6f-bff0-b2af51ff76ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" Sep 10 22:02:57.342006 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.342006 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d53242c5-034f-4b6f-bff0-b2af51ff76ac-klusterlet-config\") pod \"klusterlet-addon-workmgr-7fbccd99cb-bg2bd\" (UID: \"d53242c5-034f-4b6f-bff0-b2af51ff76ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" Sep 10 22:02:57.342241 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.342062 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2rhw\" (UniqueName: \"kubernetes.io/projected/d53242c5-034f-4b6f-bff0-b2af51ff76ac-kube-api-access-d2rhw\") pod \"klusterlet-addon-workmgr-7fbccd99cb-bg2bd\" (UID: \"d53242c5-034f-4b6f-bff0-b2af51ff76ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" Sep 10 22:02:57.342463 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.342438 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d53242c5-034f-4b6f-bff0-b2af51ff76ac-tmp\") pod \"klusterlet-addon-workmgr-7fbccd99cb-bg2bd\" (UID: \"d53242c5-034f-4b6f-bff0-b2af51ff76ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" Sep 10 22:02:57.344566 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.344538 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d53242c5-034f-4b6f-bff0-b2af51ff76ac-klusterlet-config\") pod \"klusterlet-addon-workmgr-7fbccd99cb-bg2bd\" (UID: \"d53242c5-034f-4b6f-bff0-b2af51ff76ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" Sep 10 22:02:57.350538 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.350510 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2rhw\" (UniqueName: \"kubernetes.io/projected/d53242c5-034f-4b6f-bff0-b2af51ff76ac-kube-api-access-d2rhw\") pod \"klusterlet-addon-workmgr-7fbccd99cb-bg2bd\" (UID: \"d53242c5-034f-4b6f-bff0-b2af51ff76ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" Sep 10 22:02:57.456508 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.456414 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" Sep 10 22:02:57.723066 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.722974 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd"] Sep 10 22:02:57.725942 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:02:57.725916 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd53242c5_034f_4b6f_bff0_b2af51ff76ac.slice/crio-7404374707f8cc796a0253b95d1d8421647baed250420b8f92d4a5da2187e70d WatchSource:0}: Error finding container 7404374707f8cc796a0253b95d1d8421647baed250420b8f92d4a5da2187e70d: Status 404 returned error can't find the container with id 7404374707f8cc796a0253b95d1d8421647baed250420b8f92d4a5da2187e70d Sep 10 22:02:57.916337 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:57.916301 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" event={"ID":"d53242c5-034f-4b6f-bff0-b2af51ff76ac","Type":"ContainerStarted","Data":"7404374707f8cc796a0253b95d1d8421647baed250420b8f92d4a5da2187e70d"} Sep 10 22:02:59.891081 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:02:59.891043 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hmlp2" Sep 10 22:03:00.266664 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:00.266575 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-qbvrr\" (UID: \"efc199a6-0151-4410-a431-d8652e0541f4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:03:00.269050 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:00.269022 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efc199a6-0151-4410-a431-d8652e0541f4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-qbvrr\" (UID: \"efc199a6-0151-4410-a431-d8652e0541f4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:03:00.283850 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:00.283808 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" Sep 10 22:03:01.623695 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:01.623557 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr"] Sep 10 22:03:01.627296 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:03:01.627265 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefc199a6_0151_4410_a431_d8652e0541f4.slice/crio-81df52fc04e01b32c1a1a3866ffce6e9f349949a581c6f637244160e89432153 WatchSource:0}: Error finding container 81df52fc04e01b32c1a1a3866ffce6e9f349949a581c6f637244160e89432153: Status 404 returned error can't find the container with id 81df52fc04e01b32c1a1a3866ffce6e9f349949a581c6f637244160e89432153 Sep 10 22:03:01.933048 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:01.932950 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" event={"ID":"d53242c5-034f-4b6f-bff0-b2af51ff76ac","Type":"ContainerStarted","Data":"233fd380a3f79e313896eb03b5b023c6d059eec7f3b1265daafa3fce8c06e03a"} Sep 10 22:03:01.933216 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:01.933144 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" Sep 10 22:03:01.934259 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:01.934233 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" event={"ID":"efc199a6-0151-4410-a431-d8652e0541f4","Type":"ContainerStarted","Data":"81df52fc04e01b32c1a1a3866ffce6e9f349949a581c6f637244160e89432153"} Sep 10 22:03:01.934960 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:01.934941 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" Sep 10 22:03:01.947397 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:01.947348 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fbccd99cb-bg2bd" podStartSLOduration=1.264895242 podStartE2EDuration="4.947332292s" totalImagesPullingTime="3.68243705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:02:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:03:01.947060367 +0000 UTC m=+112.068759150" watchObservedRunningTime="2026-09-10 22:03:01.947332292 +0000 UTC m=+112.069031071" Sep 10 22:03:02.314021 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:02.313937 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:02.388316 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:02.388284 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:02.446572 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:02.446538 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:03.630532 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:03.630500 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:03.796768 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:03.796732 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dw5xc"] Sep 10 22:03:03.796958 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:03.796937 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:03.799939 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:03.799919 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dw5xc" Sep 10 22:03:03.802014 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:03.801990 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 10 22:03:03.802131 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:03.801998 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-dsddp\"" Sep 10 22:03:03.809825 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:03.809784 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dw5xc"] Sep 10 22:03:03.871521 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:03.871493 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:03.899679 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:03.899596 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0f1f1d9a-4c30-43a5-8f40-2444b200fe9b-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-dw5xc\" (UID: \"0f1f1d9a-4c30-43a5-8f40-2444b200fe9b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dw5xc" Sep 10 22:03:03.944027 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:03.943989 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" event={"ID":"efc199a6-0151-4410-a431-d8652e0541f4","Type":"ContainerStarted","Data":"6a88bea872db378603e5c6d57450c5adea47018d4571a435028bf2512bce3b09"} Sep 10 22:03:03.959309 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:03.959255 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qbvrr" podStartSLOduration=34.291414739 podStartE2EDuration="35.959236556s" totalImagesPullingTime="1.667821817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:03:03.959070749 +0000 UTC m=+114.080769532" watchObservedRunningTime="2026-09-10 22:03:03.959236556 +0000 UTC m=+114.080935380" Sep 10 22:03:04.000296 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:04.000257 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0f1f1d9a-4c30-43a5-8f40-2444b200fe9b-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-dw5xc\" (UID: \"0f1f1d9a-4c30-43a5-8f40-2444b200fe9b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dw5xc" Sep 10 22:03:04.003040 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:04.003011 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0f1f1d9a-4c30-43a5-8f40-2444b200fe9b-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-dw5xc\" (UID: \"0f1f1d9a-4c30-43a5-8f40-2444b200fe9b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dw5xc" Sep 10 22:03:04.109149 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:04.109110 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dw5xc" Sep 10 22:03:04.387499 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:04.387409 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dw5xc"] Sep 10 22:03:04.390186 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:03:04.390153 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f1f1d9a_4c30_43a5_8f40_2444b200fe9b.slice/crio-bef8ddd1d9d4532db552721c93e06faf19e4eb611a77504d32f041f85d024633 WatchSource:0}: Error finding container bef8ddd1d9d4532db552721c93e06faf19e4eb611a77504d32f041f85d024633: Status 404 returned error can't find the container with id bef8ddd1d9d4532db552721c93e06faf19e4eb611a77504d32f041f85d024633 Sep 10 22:03:04.949040 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:04.949005 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dw5xc" event={"ID":"0f1f1d9a-4c30-43a5-8f40-2444b200fe9b","Type":"ContainerStarted","Data":"bef8ddd1d9d4532db552721c93e06faf19e4eb611a77504d32f041f85d024633"} Sep 10 22:03:05.953167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:05.953131 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dw5xc" event={"ID":"0f1f1d9a-4c30-43a5-8f40-2444b200fe9b","Type":"ContainerStarted","Data":"40527dd8d64d8ff1c1e5e3e8357d30721b2e57f4b08dd942045f144e49770834"} Sep 10 22:03:05.953630 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:05.953337 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dw5xc" Sep 10 22:03:05.958088 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:05.958066 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dw5xc" Sep 10 22:03:05.968263 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:05.968199 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dw5xc" podStartSLOduration=1.9110702609999999 podStartE2EDuration="2.968178858s" totalImagesPullingTime="1.057108597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:03:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:03:05.967791784 +0000 UTC m=+116.089490559" watchObservedRunningTime="2026-09-10 22:03:05.968178858 +0000 UTC m=+116.089877659" Sep 10 22:03:06.818608 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:06.818577 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:06.859171 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:06.859136 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4"] Sep 10 22:03:06.862241 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:06.862220 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" Sep 10 22:03:06.864408 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:06.864384 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 10 22:03:06.864537 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:06.864392 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 10 22:03:06.865074 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:06.865055 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dbmtq\"" Sep 10 22:03:06.865074 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:06.865073 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 22:03:06.869539 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:06.869517 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4"] Sep 10 22:03:06.886972 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:06.886941 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:07.025759 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.025712 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/849fcec6-8056-479e-bf59-55d371abb6a1-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-mqrg4\" (UID: \"849fcec6-8056-479e-bf59-55d371abb6a1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" Sep 10 22:03:07.026232 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.025766 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/849fcec6-8056-479e-bf59-55d371abb6a1-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-mqrg4\" (UID: \"849fcec6-8056-479e-bf59-55d371abb6a1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" Sep 10 22:03:07.026232 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.025871 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/849fcec6-8056-479e-bf59-55d371abb6a1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-mqrg4\" (UID: \"849fcec6-8056-479e-bf59-55d371abb6a1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" Sep 10 22:03:07.026232 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.026000 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pg4z\" (UniqueName: \"kubernetes.io/projected/849fcec6-8056-479e-bf59-55d371abb6a1-kube-api-access-4pg4z\") pod \"prometheus-operator-68bcd7f987-mqrg4\" (UID: \"849fcec6-8056-479e-bf59-55d371abb6a1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" Sep 10 22:03:07.126563 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.126454 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pg4z\" (UniqueName: \"kubernetes.io/projected/849fcec6-8056-479e-bf59-55d371abb6a1-kube-api-access-4pg4z\") pod \"prometheus-operator-68bcd7f987-mqrg4\" (UID: \"849fcec6-8056-479e-bf59-55d371abb6a1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" Sep 10 22:03:07.126563 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.126529 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/849fcec6-8056-479e-bf59-55d371abb6a1-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-mqrg4\" (UID: \"849fcec6-8056-479e-bf59-55d371abb6a1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" Sep 10 22:03:07.126563 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.126547 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/849fcec6-8056-479e-bf59-55d371abb6a1-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-mqrg4\" (UID: \"849fcec6-8056-479e-bf59-55d371abb6a1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" Sep 10 22:03:07.126847 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.126568 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/849fcec6-8056-479e-bf59-55d371abb6a1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-mqrg4\" (UID: \"849fcec6-8056-479e-bf59-55d371abb6a1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" Sep 10 22:03:07.126847 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:03:07.126665 2576 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 10 22:03:07.126847 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:03:07.126790 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/849fcec6-8056-479e-bf59-55d371abb6a1-prometheus-operator-tls podName:849fcec6-8056-479e-bf59-55d371abb6a1 nodeName:}" failed. No retries permitted until 2026-09-10 22:03:07.626763607 +0000 UTC m=+117.748462366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/849fcec6-8056-479e-bf59-55d371abb6a1-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-mqrg4" (UID: "849fcec6-8056-479e-bf59-55d371abb6a1") : secret "prometheus-operator-tls" not found Sep 10 22:03:07.127243 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.127224 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/849fcec6-8056-479e-bf59-55d371abb6a1-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-mqrg4\" (UID: \"849fcec6-8056-479e-bf59-55d371abb6a1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" Sep 10 22:03:07.128861 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.128843 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/849fcec6-8056-479e-bf59-55d371abb6a1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-mqrg4\" (UID: \"849fcec6-8056-479e-bf59-55d371abb6a1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" Sep 10 22:03:07.135075 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.135042 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pg4z\" (UniqueName: \"kubernetes.io/projected/849fcec6-8056-479e-bf59-55d371abb6a1-kube-api-access-4pg4z\") pod \"prometheus-operator-68bcd7f987-mqrg4\" (UID: \"849fcec6-8056-479e-bf59-55d371abb6a1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" Sep 10 22:03:07.153990 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.153952 2576 patch_prober.go:28] interesting pod/image-registry-6cdb65cf65-bnlmv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 10 22:03:07.154142 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.154005 2576 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" podUID="31f47884-061b-4eb7-b27b-91d855d9bf3a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 22:03:07.197786 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.197752 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:07.630236 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.630199 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/849fcec6-8056-479e-bf59-55d371abb6a1-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-mqrg4\" (UID: \"849fcec6-8056-479e-bf59-55d371abb6a1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" Sep 10 22:03:07.632861 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.632837 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/849fcec6-8056-479e-bf59-55d371abb6a1-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-mqrg4\" (UID: \"849fcec6-8056-479e-bf59-55d371abb6a1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" Sep 10 22:03:07.771911 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:07.771869 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" Sep 10 22:03:08.044585 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:08.044561 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4"] Sep 10 22:03:08.046824 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:03:08.046781 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod849fcec6_8056_479e_bf59_55d371abb6a1.slice/crio-2baa54549be1defce91b234d2bdd5599cfa43fb682409aa06f57b0db7bf4f6d9 WatchSource:0}: Error finding container 2baa54549be1defce91b234d2bdd5599cfa43fb682409aa06f57b0db7bf4f6d9: Status 404 returned error can't find the container with id 2baa54549be1defce91b234d2bdd5599cfa43fb682409aa06f57b0db7bf4f6d9 Sep 10 22:03:08.884752 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:08.884719 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:03:08.966908 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:08.966862 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" event={"ID":"849fcec6-8056-479e-bf59-55d371abb6a1","Type":"ContainerStarted","Data":"2baa54549be1defce91b234d2bdd5599cfa43fb682409aa06f57b0db7bf4f6d9"} Sep 10 22:03:09.028234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:09.028201 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:09.128410 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:09.128381 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:09.231884 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:09.231855 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:09.373519 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:09.373489 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:09.769304 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:09.769274 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:09.835340 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:09.835312 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:09.971621 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:09.971589 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" event={"ID":"849fcec6-8056-479e-bf59-55d371abb6a1","Type":"ContainerStarted","Data":"cc42814d845b306a0710ce294dad449dabe64dd30ecd9d14ea4245bdb66023c7"} Sep 10 22:03:09.971779 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:09.971641 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" event={"ID":"849fcec6-8056-479e-bf59-55d371abb6a1","Type":"ContainerStarted","Data":"46c53f87838127a896d2e992445d56cfd90ea69f9e11c94ca4c075e4c0101be1"} Sep 10 22:03:09.988844 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:09.988762 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-mqrg4" podStartSLOduration=2.931160501 podStartE2EDuration="3.988739574s" totalImagesPullingTime="1.057579073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:03:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:03:09.988625754 +0000 UTC m=+120.110324550" watchObservedRunningTime="2026-09-10 22:03:09.988739574 +0000 UTC m=+120.110438357" Sep 10 22:03:11.988888 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:11.988854 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:12.247687 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.247603 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:12.369424 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.369387 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rvclz"] Sep 10 22:03:12.373103 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.373082 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.380183 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.380158 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 22:03:12.380910 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.380890 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7v8kf\"" Sep 10 22:03:12.381646 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.381630 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 22:03:12.384159 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.384140 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 22:03:12.389010 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.388981 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg"] Sep 10 22:03:12.392433 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.392414 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.397533 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.397512 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 10 22:03:12.398274 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.398248 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 10 22:03:12.398601 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.398552 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-6c855\"" Sep 10 22:03:12.398850 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.398816 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 10 22:03:12.415874 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.415835 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg"] Sep 10 22:03:12.437813 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.437781 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:12.468226 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.468194 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-node-exporter-tls\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.468396 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.468235 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-node-exporter-accelerators-collector-config\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.468396 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.468270 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-root\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.468396 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.468303 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-metrics-client-ca\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.468396 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.468344 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfgkr\" (UniqueName: \"kubernetes.io/projected/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-kube-api-access-mfgkr\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.468396 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.468363 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-node-exporter-wtmp\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.468396 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.468378 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-sys\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.468579 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.468453 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.468579 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.468480 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-node-exporter-textfile\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.562289 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.562194 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:12.569176 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569142 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-node-exporter-tls\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.569336 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569188 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-node-exporter-accelerators-collector-config\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.569336 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569227 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-root\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.569336 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569242 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-metrics-client-ca\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.569336 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569265 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.569336 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569292 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfgkr\" (UniqueName: \"kubernetes.io/projected/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-kube-api-access-mfgkr\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.569617 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569344 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-node-exporter-wtmp\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.569617 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569396 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.569617 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569440 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.569617 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569460 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-root\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.569617 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569484 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-sys\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.569617 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569525 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-sys\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.569617 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569529 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.569617 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569553 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-node-exporter-wtmp\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.569617 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569605 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.570114 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569686 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.570114 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569719 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-node-exporter-textfile\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.570114 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569743 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqk77\" (UniqueName: \"kubernetes.io/projected/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-kube-api-access-mqk77\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.570114 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.569970 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-metrics-client-ca\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.570114 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.570065 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-node-exporter-textfile\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.570114 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.570085 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-node-exporter-accelerators-collector-config\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.571748 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.571721 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-node-exporter-tls\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.572140 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.572120 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.577834 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.577794 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfgkr\" (UniqueName: \"kubernetes.io/projected/d5d5cd61-7ac6-4def-bcd7-883ababaa59a-kube-api-access-mfgkr\") pod \"node-exporter-rvclz\" (UID: \"d5d5cd61-7ac6-4def-bcd7-883ababaa59a\") " pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.661612 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.661577 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:12.670684 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.670647 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqk77\" (UniqueName: \"kubernetes.io/projected/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-kube-api-access-mqk77\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.670866 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.670838 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.670928 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.670876 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.670928 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.670893 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.671041 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.670926 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.671041 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.670955 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.671041 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:03:12.671014 2576 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Sep 10 22:03:12.671175 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:03:12.671108 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-kube-state-metrics-tls podName:b7ba4ac6-2237-4cd3-95d9-c23cdea61151 nodeName:}" failed. No retries permitted until 2026-09-10 22:03:13.171069038 +0000 UTC m=+123.292767813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-nswfg" (UID: "b7ba4ac6-2237-4cd3-95d9-c23cdea61151") : secret "kube-state-metrics-tls" not found Sep 10 22:03:12.671397 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.671374 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.671603 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.671580 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.671641 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.671598 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.673370 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.673353 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.681050 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.681029 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqk77\" (UniqueName: \"kubernetes.io/projected/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-kube-api-access-mqk77\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:12.681818 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.681788 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rvclz" Sep 10 22:03:12.693518 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:03:12.693487 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d5cd61_7ac6_4def_bcd7_883ababaa59a.slice/crio-e411ef8c30933b76da97f67a405bd1e9db586187a5d96c5b8cc519e1917f4143 WatchSource:0}: Error finding container e411ef8c30933b76da97f67a405bd1e9db586187a5d96c5b8cc519e1917f4143: Status 404 returned error can't find the container with id e411ef8c30933b76da97f67a405bd1e9db586187a5d96c5b8cc519e1917f4143 Sep 10 22:03:12.765731 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.765705 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:12.980459 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.980414 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:12.983040 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:12.983008 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rvclz" event={"ID":"d5d5cd61-7ac6-4def-bcd7-883ababaa59a","Type":"ContainerStarted","Data":"e411ef8c30933b76da97f67a405bd1e9db586187a5d96c5b8cc519e1917f4143"} Sep 10 22:03:13.176248 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.176213 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:13.179335 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.179307 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7ba4ac6-2237-4cd3-95d9-c23cdea61151-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-nswfg\" (UID: \"b7ba4ac6-2237-4cd3-95d9-c23cdea61151\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:13.297187 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.297107 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:13.301712 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.301679 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" Sep 10 22:03:13.401435 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.401285 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:13.419847 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.419814 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:03:13.423566 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.423537 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.425861 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.425835 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 22:03:13.426022 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.425835 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 22:03:13.426022 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.425835 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 22:03:13.426022 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.425869 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 22:03:13.426280 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.426261 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 22:03:13.426332 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.426297 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 22:03:13.426332 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.426304 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 22:03:13.427234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.427177 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 22:03:13.427341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.427278 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-v2jpn\"" Sep 10 22:03:13.428205 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.428106 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 22:03:13.439641 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.439542 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:03:13.580463 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.580423 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-config-volume\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.580631 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.580475 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/024917fc-4a5d-4cd5-9b26-1ec98de340ed-config-out\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.580631 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.580521 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-web-config\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.580631 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.580548 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/024917fc-4a5d-4cd5-9b26-1ec98de340ed-tls-assets\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.580631 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.580572 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/024917fc-4a5d-4cd5-9b26-1ec98de340ed-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.580631 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.580621 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/024917fc-4a5d-4cd5-9b26-1ec98de340ed-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.580843 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.580659 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.580843 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.580688 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfn8x\" (UniqueName: \"kubernetes.io/projected/024917fc-4a5d-4cd5-9b26-1ec98de340ed-kube-api-access-wfn8x\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.580843 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.580722 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.580843 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.580749 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.580843 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.580789 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.581054 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.580853 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/024917fc-4a5d-4cd5-9b26-1ec98de340ed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.581054 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.580882 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.599305 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.599264 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg"] Sep 10 22:03:13.641787 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:03:13.641743 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ba4ac6_2237_4cd3_95d9_c23cdea61151.slice/crio-5174f26b0b8fcabea06f8559fb26af2d4aa73849413697f6deb98acf15633909 WatchSource:0}: Error finding container 5174f26b0b8fcabea06f8559fb26af2d4aa73849413697f6deb98acf15633909: Status 404 returned error can't find the container with id 5174f26b0b8fcabea06f8559fb26af2d4aa73849413697f6deb98acf15633909 Sep 10 22:03:13.682344 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.682312 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-config-volume\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.682344 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.682348 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/024917fc-4a5d-4cd5-9b26-1ec98de340ed-config-out\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.682591 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.682391 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-web-config\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.682591 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.682418 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/024917fc-4a5d-4cd5-9b26-1ec98de340ed-tls-assets\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.682591 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.682441 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/024917fc-4a5d-4cd5-9b26-1ec98de340ed-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.682591 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.682484 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/024917fc-4a5d-4cd5-9b26-1ec98de340ed-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.683011 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.682683 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.683011 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.682723 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfn8x\" (UniqueName: \"kubernetes.io/projected/024917fc-4a5d-4cd5-9b26-1ec98de340ed-kube-api-access-wfn8x\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.683011 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.682860 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.683011 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.682966 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.683214 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.683029 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.683214 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.683089 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/024917fc-4a5d-4cd5-9b26-1ec98de340ed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.683214 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.683119 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.683214 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:03:13.683206 2576 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 10 22:03:13.683404 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:03:13.683284 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-main-tls podName:024917fc-4a5d-4cd5-9b26-1ec98de340ed nodeName:}" failed. No retries permitted until 2026-09-10 22:03:14.183263722 +0000 UTC m=+124.304962498 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "024917fc-4a5d-4cd5-9b26-1ec98de340ed") : secret "alertmanager-main-tls" not found Sep 10 22:03:13.683787 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.683745 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/024917fc-4a5d-4cd5-9b26-1ec98de340ed-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.684654 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.684626 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/024917fc-4a5d-4cd5-9b26-1ec98de340ed-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.685753 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.685723 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.686187 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.686121 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/024917fc-4a5d-4cd5-9b26-1ec98de340ed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.686883 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.686858 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.687934 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.687449 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.687934 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.687894 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-config-volume\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.688177 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.688153 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-web-config\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.688638 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.688329 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.688763 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.688741 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/024917fc-4a5d-4cd5-9b26-1ec98de340ed-config-out\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.688997 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.688975 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/024917fc-4a5d-4cd5-9b26-1ec98de340ed-tls-assets\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.696575 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.696535 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfn8x\" (UniqueName: \"kubernetes.io/projected/024917fc-4a5d-4cd5-9b26-1ec98de340ed-kube-api-access-wfn8x\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:13.772792 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.772717 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:13.988638 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.988602 2576 generic.go:356] "Generic (PLEG): container finished" podID="d5d5cd61-7ac6-4def-bcd7-883ababaa59a" containerID="071abfaeb227071edf4e12a7467ff8ccef2bff60d34b4476b2714357d09b7774" exitCode=0 Sep 10 22:03:13.988853 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.988697 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rvclz" event={"ID":"d5d5cd61-7ac6-4def-bcd7-883ababaa59a","Type":"ContainerDied","Data":"071abfaeb227071edf4e12a7467ff8ccef2bff60d34b4476b2714357d09b7774"} Sep 10 22:03:13.990083 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:13.990058 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" event={"ID":"b7ba4ac6-2237-4cd3-95d9-c23cdea61151","Type":"ContainerStarted","Data":"5174f26b0b8fcabea06f8559fb26af2d4aa73849413697f6deb98acf15633909"} Sep 10 22:03:14.187904 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:14.187873 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:14.190962 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:14.190937 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:14.335608 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:14.335569 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:03:14.625009 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:14.624978 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:03:14.737260 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:03:14.737181 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod024917fc_4a5d_4cd5_9b26_1ec98de340ed.slice/crio-d2b164bda1f4e12e181c321eb93279510e330c82ef873a9eab4ad2beb39aa0cc WatchSource:0}: Error finding container d2b164bda1f4e12e181c321eb93279510e330c82ef873a9eab4ad2beb39aa0cc: Status 404 returned error can't find the container with id d2b164bda1f4e12e181c321eb93279510e330c82ef873a9eab4ad2beb39aa0cc Sep 10 22:03:14.996347 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:14.996171 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"024917fc-4a5d-4cd5-9b26-1ec98de340ed","Type":"ContainerStarted","Data":"d2b164bda1f4e12e181c321eb93279510e330c82ef873a9eab4ad2beb39aa0cc"} Sep 10 22:03:14.998827 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:14.998780 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rvclz" event={"ID":"d5d5cd61-7ac6-4def-bcd7-883ababaa59a","Type":"ContainerStarted","Data":"aa6cf455212ca9887f9b49ab56868800c08b16dd2a4f693472dcf3136a4e6791"} Sep 10 22:03:14.998969 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:14.998851 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rvclz" event={"ID":"d5d5cd61-7ac6-4def-bcd7-883ababaa59a","Type":"ContainerStarted","Data":"dc13431beb525b031a132d9113f0dd7134feed08512174bb30e4aa2a39fd408c"} Sep 10 22:03:15.001366 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:15.001295 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" event={"ID":"b7ba4ac6-2237-4cd3-95d9-c23cdea61151","Type":"ContainerStarted","Data":"e288bc5254aaac2a556017f764b1cdbeba8a9d0266caa6a519de94f4af6a1357"} Sep 10 22:03:15.001366 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:15.001350 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" event={"ID":"b7ba4ac6-2237-4cd3-95d9-c23cdea61151","Type":"ContainerStarted","Data":"071cc7a1b56a62202bed62568e179da7d1106c414e84e572ad031ef5c0879b1e"} Sep 10 22:03:15.001580 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:15.001378 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" event={"ID":"b7ba4ac6-2237-4cd3-95d9-c23cdea61151","Type":"ContainerStarted","Data":"cf1cbac3df225ad338861aa6810a1e8ea31da9c2b94a95d9e1a46d0348973763"} Sep 10 22:03:15.015516 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:15.015446 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rvclz" podStartSLOduration=1.9525727499999999 podStartE2EDuration="3.015423832s" totalImagesPullingTime="995.407227ms" totalInitContainerRuntime="67.443855ms" isStatefulPod=true podCreationTimestamp="2026-09-10 22:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:03:15.015259251 +0000 UTC m=+125.136958035" watchObservedRunningTime="2026-09-10 22:03:15.015423832 +0000 UTC m=+125.137122615" Sep 10 22:03:15.033164 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:15.033100 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nswfg" podStartSLOduration=1.888353687 podStartE2EDuration="3.033075491s" totalImagesPullingTime="1.144721804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:03:15.032531399 +0000 UTC m=+125.154230205" watchObservedRunningTime="2026-09-10 22:03:15.033075491 +0000 UTC m=+125.154774274" Sep 10 22:03:15.153233 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:15.153197 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:15.254474 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:15.254373 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:15.354361 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:15.354315 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:15.566740 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:15.566705 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:15.667238 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:15.667214 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:16.006180 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:16.006090 2576 generic.go:356] "Generic (PLEG): container finished" podID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerID="80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7" exitCode=0 Sep 10 22:03:16.006180 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:16.006165 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"024917fc-4a5d-4cd5-9b26-1ec98de340ed","Type":"ContainerDied","Data":"80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7"} Sep 10 22:03:17.002914 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:17.001605 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj"] Sep 10 22:03:17.004522 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:17.004495 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:17.005288 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:17.005265 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj" Sep 10 22:03:17.009425 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:17.009404 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-csftn\"" Sep 10 22:03:17.009628 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:17.009604 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 10 22:03:17.017225 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:17.017201 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj"] Sep 10 22:03:17.017427 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:17.017408 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk2h6\" (UniqueName: \"kubernetes.io/projected/41258563-033c-42e0-a51e-4e77fb6f11c6-kube-api-access-tk2h6\") pod \"monitoring-plugin-5d85778b9d-wswcj\" (UID: \"41258563-033c-42e0-a51e-4e77fb6f11c6\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj" Sep 10 22:03:17.017499 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:17.017460 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/41258563-033c-42e0-a51e-4e77fb6f11c6-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-wswcj\" (UID: \"41258563-033c-42e0-a51e-4e77fb6f11c6\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj" Sep 10 22:03:17.118118 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:17.118078 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk2h6\" (UniqueName: \"kubernetes.io/projected/41258563-033c-42e0-a51e-4e77fb6f11c6-kube-api-access-tk2h6\") pod \"monitoring-plugin-5d85778b9d-wswcj\" (UID: \"41258563-033c-42e0-a51e-4e77fb6f11c6\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj" Sep 10 22:03:17.118333 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:17.118140 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/41258563-033c-42e0-a51e-4e77fb6f11c6-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-wswcj\" (UID: \"41258563-033c-42e0-a51e-4e77fb6f11c6\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj" Sep 10 22:03:17.118333 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:03:17.118250 2576 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 10 22:03:17.118333 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:03:17.118317 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41258563-033c-42e0-a51e-4e77fb6f11c6-monitoring-plugin-cert podName:41258563-033c-42e0-a51e-4e77fb6f11c6 nodeName:}" failed. No retries permitted until 2026-09-10 22:03:17.618297757 +0000 UTC m=+127.739996532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/41258563-033c-42e0-a51e-4e77fb6f11c6-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-wswcj" (UID: "41258563-033c-42e0-a51e-4e77fb6f11c6") : secret "monitoring-plugin-cert" not found Sep 10 22:03:17.129059 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:17.129027 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk2h6\" (UniqueName: \"kubernetes.io/projected/41258563-033c-42e0-a51e-4e77fb6f11c6-kube-api-access-tk2h6\") pod \"monitoring-plugin-5d85778b9d-wswcj\" (UID: \"41258563-033c-42e0-a51e-4e77fb6f11c6\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj" Sep 10 22:03:17.198029 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:17.197990 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:17.622442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:17.622409 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/41258563-033c-42e0-a51e-4e77fb6f11c6-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-wswcj\" (UID: \"41258563-033c-42e0-a51e-4e77fb6f11c6\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj" Sep 10 22:03:17.624729 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:17.624705 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/41258563-033c-42e0-a51e-4e77fb6f11c6-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-wswcj\" (UID: \"41258563-033c-42e0-a51e-4e77fb6f11c6\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj" Sep 10 22:03:17.917402 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:17.917352 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj" Sep 10 22:03:18.020293 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.020245 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"024917fc-4a5d-4cd5-9b26-1ec98de340ed","Type":"ContainerStarted","Data":"960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a"} Sep 10 22:03:18.020716 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.020307 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"024917fc-4a5d-4cd5-9b26-1ec98de340ed","Type":"ContainerStarted","Data":"011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7"} Sep 10 22:03:18.020716 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.020335 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"024917fc-4a5d-4cd5-9b26-1ec98de340ed","Type":"ContainerStarted","Data":"c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259"} Sep 10 22:03:18.020716 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.020359 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"024917fc-4a5d-4cd5-9b26-1ec98de340ed","Type":"ContainerStarted","Data":"3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef"} Sep 10 22:03:18.020716 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.020383 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"024917fc-4a5d-4cd5-9b26-1ec98de340ed","Type":"ContainerStarted","Data":"e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073"} Sep 10 22:03:18.201294 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.201204 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj"] Sep 10 22:03:18.257959 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:03:18.257916 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41258563_033c_42e0_a51e_4e77fb6f11c6.slice/crio-ae06277fc9c3ee887bea70bc49bec2b1cb7293184f67dcaec8f3e0eb91124448 WatchSource:0}: Error finding container ae06277fc9c3ee887bea70bc49bec2b1cb7293184f67dcaec8f3e0eb91124448: Status 404 returned error can't find the container with id ae06277fc9c3ee887bea70bc49bec2b1cb7293184f67dcaec8f3e0eb91124448 Sep 10 22:03:18.509319 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.509228 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:18.518385 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.518353 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 22:03:18.522339 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.522306 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.524593 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.524567 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 22:03:18.525114 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.525096 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 22:03:18.525193 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.525096 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 22:03:18.525523 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.525378 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6v9zt\"" Sep 10 22:03:18.525523 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.525400 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dqeor0lierrf3\"" Sep 10 22:03:18.525523 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.525404 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 22:03:18.525523 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.525457 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 22:03:18.525523 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.525462 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 22:03:18.526092 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.526070 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 22:03:18.526198 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.526106 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 22:03:18.526198 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.526129 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 22:03:18.526359 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.526341 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 22:03:18.526426 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.526361 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 22:03:18.526673 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.526080 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 22:03:18.526673 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.526634 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 22:03:18.528962 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.528940 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 22:03:18.531315 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.531248 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 22:03:18.532131 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532112 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f13be60c-9156-4e4a-a067-7764705e6564-config-out\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532207 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532137 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532207 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532153 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532207 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532170 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532330 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532195 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htmql\" (UniqueName: \"kubernetes.io/projected/f13be60c-9156-4e4a-a067-7764705e6564-kube-api-access-htmql\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532330 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532244 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532330 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532281 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532330 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532326 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532448 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532343 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-config\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532448 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532379 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532448 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532410 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f13be60c-9156-4e4a-a067-7764705e6564-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532563 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532461 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532563 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532506 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532563 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532542 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532716 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532601 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532716 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532624 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-web-config\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532716 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532650 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532716 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532687 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532716 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532713 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.532990 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.532738 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.540363 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.540334 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 22:03:18.633284 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633243 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633284 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633289 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-web-config\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633555 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633316 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633555 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633347 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633555 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633372 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633555 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633398 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633555 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633420 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f13be60c-9156-4e4a-a067-7764705e6564-config-out\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633555 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633442 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633574 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633618 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633643 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htmql\" (UniqueName: \"kubernetes.io/projected/f13be60c-9156-4e4a-a067-7764705e6564-kube-api-access-htmql\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633671 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633707 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633751 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633782 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-config\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633843 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.633877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633872 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f13be60c-9156-4e4a-a067-7764705e6564-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.634307 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633907 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.634307 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.633965 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.634307 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.634045 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.634458 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.634340 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.634458 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.634340 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.635771 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.634968 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.637864 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.636628 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.637864 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.636645 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.637864 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.636976 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.637864 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.637177 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.637864 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.637708 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.637864 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.637719 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.638256 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.637875 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.638256 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.638152 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-web-config\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.638359 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.638278 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.638359 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.638296 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-config\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.638485 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.638465 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f13be60c-9156-4e4a-a067-7764705e6564-config-out\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.639512 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.639465 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f13be60c-9156-4e4a-a067-7764705e6564-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.639609 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.639540 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.640229 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.640208 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.640974 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.640955 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.641344 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.641326 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.644634 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.644607 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htmql\" (UniqueName: \"kubernetes.io/projected/f13be60c-9156-4e4a-a067-7764705e6564-kube-api-access-htmql\") pod \"prometheus-k8s-0\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.835544 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.835377 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:18.895680 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.894220 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:18.959675 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:18.959632 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:19.026429 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:19.026387 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj" event={"ID":"41258563-033c-42e0-a51e-4e77fb6f11c6","Type":"ContainerStarted","Data":"ae06277fc9c3ee887bea70bc49bec2b1cb7293184f67dcaec8f3e0eb91124448"} Sep 10 22:03:19.030387 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:19.030345 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"024917fc-4a5d-4cd5-9b26-1ec98de340ed","Type":"ContainerStarted","Data":"5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a"} Sep 10 22:03:19.058606 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:19.058547 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.126550019 podStartE2EDuration="6.058528713s" totalImagesPullingTime="2.911658184s" totalInitContainerRuntime="20.32051ms" isStatefulPod=false podCreationTimestamp="2026-09-10 22:03:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:03:19.056642013 +0000 UTC m=+129.178340795" watchObservedRunningTime="2026-09-10 22:03:19.058528713 +0000 UTC m=+129.180227495" Sep 10 22:03:19.127380 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:19.127311 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 22:03:19.229744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:19.229701 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cdb65cf65-bnlmv"] Sep 10 22:03:19.319352 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:19.319314 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:19.395402 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:03:19.395314 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13be60c_9156_4e4a_a067_7764705e6564.slice/crio-5dac3eaf47feb38bc6035a2232ff9b771beadd9584bd014ed31323b4dbfab96a WatchSource:0}: Error finding container 5dac3eaf47feb38bc6035a2232ff9b771beadd9584bd014ed31323b4dbfab96a: Status 404 returned error can't find the container with id 5dac3eaf47feb38bc6035a2232ff9b771beadd9584bd014ed31323b4dbfab96a Sep 10 22:03:19.421140 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:19.421114 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:20.036472 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:20.036431 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj" event={"ID":"41258563-033c-42e0-a51e-4e77fb6f11c6","Type":"ContainerStarted","Data":"267526b800e2f121e43d20ad3db9803936a299317d1e740e5fb66fc587463553"} Sep 10 22:03:20.036931 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:20.036499 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj" Sep 10 22:03:20.038245 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:20.038213 2576 generic.go:356] "Generic (PLEG): container finished" podID="f13be60c-9156-4e4a-a067-7764705e6564" containerID="fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815" exitCode=0 Sep 10 22:03:20.038380 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:20.038247 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f13be60c-9156-4e4a-a067-7764705e6564","Type":"ContainerDied","Data":"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815"} Sep 10 22:03:20.038380 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:20.038314 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f13be60c-9156-4e4a-a067-7764705e6564","Type":"ContainerStarted","Data":"5dac3eaf47feb38bc6035a2232ff9b771beadd9584bd014ed31323b4dbfab96a"} Sep 10 22:03:20.041784 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:20.041759 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj" Sep 10 22:03:20.056270 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:20.056215 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-wswcj" podStartSLOduration=2.873833798 podStartE2EDuration="4.056198276s" totalImagesPullingTime="1.182364478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:03:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:03:20.055537047 +0000 UTC m=+130.177235829" watchObservedRunningTime="2026-09-10 22:03:20.056198276 +0000 UTC m=+130.177897070" Sep 10 22:03:20.147769 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:20.147613 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs\") pod \"network-metrics-daemon-lvd5t\" (UID: \"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0\") " pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:03:20.151222 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:20.151189 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0-metrics-certs\") pod \"network-metrics-daemon-lvd5t\" (UID: \"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0\") " pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:03:20.446680 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:20.446647 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vfb6l\"" Sep 10 22:03:20.454378 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:20.454342 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvd5t" Sep 10 22:03:20.738142 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:20.738120 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lvd5t"] Sep 10 22:03:20.740494 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:03:20.740453 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc4d9d9_2b45_41e5_adfb_10e9c517f9d0.slice/crio-22be8a14c9ddd47f3d39c8bab4829652dcec1623a433f61a6989f6a8dbd907aa WatchSource:0}: Error finding container 22be8a14c9ddd47f3d39c8bab4829652dcec1623a433f61a6989f6a8dbd907aa: Status 404 returned error can't find the container with id 22be8a14c9ddd47f3d39c8bab4829652dcec1623a433f61a6989f6a8dbd907aa Sep 10 22:03:21.044986 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:21.044899 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lvd5t" event={"ID":"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0","Type":"ContainerStarted","Data":"22be8a14c9ddd47f3d39c8bab4829652dcec1623a433f61a6989f6a8dbd907aa"} Sep 10 22:03:21.169265 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:21.169230 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:21.269958 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:21.269926 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:22.053222 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:22.053182 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lvd5t" event={"ID":"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0","Type":"ContainerStarted","Data":"14c0dff430eafade1c447e499c58eaed0da2cb82b6940f13464e82e9b2b93ad8"} Sep 10 22:03:23.060376 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:23.060333 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lvd5t" event={"ID":"6bc4d9d9-2b45-41e5-adfb-10e9c517f9d0","Type":"ContainerStarted","Data":"24ccc05a02b7e79cb280be471525a23d679e96e721efd47c8a6b14479c1c3d93"} Sep 10 22:03:23.082773 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:23.079589 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lvd5t" podStartSLOduration=131.903359148 podStartE2EDuration="2m13.079567186s" totalImagesPullingTime="1.176208038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:03:23.076878595 +0000 UTC m=+133.198577377" watchObservedRunningTime="2026-09-10 22:03:23.079567186 +0000 UTC m=+133.201265969" Sep 10 22:03:23.186530 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:23.186500 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:23.288619 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:23.288585 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:24.065865 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:24.065829 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f13be60c-9156-4e4a-a067-7764705e6564","Type":"ContainerStarted","Data":"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1"} Sep 10 22:03:24.066238 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:24.065926 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f13be60c-9156-4e4a-a067-7764705e6564","Type":"ContainerStarted","Data":"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990"} Sep 10 22:03:25.584558 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:25.584519 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:25.644990 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:25.644953 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:25.705284 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:25.705254 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:26.066650 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:26.066593 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:26.076066 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:26.076034 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f13be60c-9156-4e4a-a067-7764705e6564","Type":"ContainerStarted","Data":"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227"} Sep 10 22:03:26.076200 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:26.076102 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f13be60c-9156-4e4a-a067-7764705e6564","Type":"ContainerStarted","Data":"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245"} Sep 10 22:03:26.076200 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:26.076133 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f13be60c-9156-4e4a-a067-7764705e6564","Type":"ContainerStarted","Data":"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c"} Sep 10 22:03:26.076200 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:26.076162 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f13be60c-9156-4e4a-a067-7764705e6564","Type":"ContainerStarted","Data":"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff"} Sep 10 22:03:26.106667 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:26.106452 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.539870588 podStartE2EDuration="8.106414603s" totalImagesPullingTime="5.543618849s" totalInitContainerRuntime="22.925166ms" isStatefulPod=false podCreationTimestamp="2026-09-10 22:03:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:03:26.103553387 +0000 UTC m=+136.225252172" watchObservedRunningTime="2026-09-10 22:03:26.106414603 +0000 UTC m=+136.228113382" Sep 10 22:03:26.240178 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:26.240140 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:26.320209 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:26.320174 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:26.444392 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:26.444301 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:27.072861 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:27.072834 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:27.249091 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:27.249048 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:27.878281 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:27.878239 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:27.979851 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:27.979792 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:28.836532 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:28.836487 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:03:31.238651 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:31.238611 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:31.321501 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:31.321469 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:31.909620 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:31.909588 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:31.965495 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:31.965461 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:35.381284 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:35.381244 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:40.498691 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:40.498656 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:40.600075 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:40.600042 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:40.654001 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:40.653975 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:40.877065 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:40.877037 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:40.932385 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:40.932354 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:44.252100 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.252055 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" podUID="31f47884-061b-4eb7-b27b-91d855d9bf3a" containerName="registry" containerID="cri-o://1267fd43037e4ef56559407234c946bf3b17f2730e50a965ea4b99e04951e8c8" gracePeriod=30 Sep 10 22:03:44.487657 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.487629 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:03:44.559208 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.559120 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6bbq\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-kube-api-access-d6bbq\") pod \"31f47884-061b-4eb7-b27b-91d855d9bf3a\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " Sep 10 22:03:44.559208 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.559164 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/31f47884-061b-4eb7-b27b-91d855d9bf3a-ca-trust-extracted\") pod \"31f47884-061b-4eb7-b27b-91d855d9bf3a\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " Sep 10 22:03:44.559208 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.559190 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-certificates\") pod \"31f47884-061b-4eb7-b27b-91d855d9bf3a\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " Sep 10 22:03:44.559488 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.559223 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/31f47884-061b-4eb7-b27b-91d855d9bf3a-image-registry-private-configuration\") pod \"31f47884-061b-4eb7-b27b-91d855d9bf3a\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " Sep 10 22:03:44.559488 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.559260 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/31f47884-061b-4eb7-b27b-91d855d9bf3a-installation-pull-secrets\") pod \"31f47884-061b-4eb7-b27b-91d855d9bf3a\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " Sep 10 22:03:44.559488 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.559369 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/31f47884-061b-4eb7-b27b-91d855d9bf3a-trusted-ca\") pod \"31f47884-061b-4eb7-b27b-91d855d9bf3a\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " Sep 10 22:03:44.559488 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.559413 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls\") pod \"31f47884-061b-4eb7-b27b-91d855d9bf3a\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " Sep 10 22:03:44.559670 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.559489 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-bound-sa-token\") pod \"31f47884-061b-4eb7-b27b-91d855d9bf3a\" (UID: \"31f47884-061b-4eb7-b27b-91d855d9bf3a\") " Sep 10 22:03:44.559745 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.559717 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "31f47884-061b-4eb7-b27b-91d855d9bf3a" (UID: "31f47884-061b-4eb7-b27b-91d855d9bf3a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:03:44.559923 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.559892 2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-certificates\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:03:44.560025 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.559930 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31f47884-061b-4eb7-b27b-91d855d9bf3a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "31f47884-061b-4eb7-b27b-91d855d9bf3a" (UID: "31f47884-061b-4eb7-b27b-91d855d9bf3a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:03:44.561852 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.561789 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31f47884-061b-4eb7-b27b-91d855d9bf3a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "31f47884-061b-4eb7-b27b-91d855d9bf3a" (UID: "31f47884-061b-4eb7-b27b-91d855d9bf3a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:03:44.561852 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.561789 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "31f47884-061b-4eb7-b27b-91d855d9bf3a" (UID: "31f47884-061b-4eb7-b27b-91d855d9bf3a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:03:44.562032 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.561949 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31f47884-061b-4eb7-b27b-91d855d9bf3a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "31f47884-061b-4eb7-b27b-91d855d9bf3a" (UID: "31f47884-061b-4eb7-b27b-91d855d9bf3a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:03:44.562070 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.562048 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "31f47884-061b-4eb7-b27b-91d855d9bf3a" (UID: "31f47884-061b-4eb7-b27b-91d855d9bf3a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:03:44.562173 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.562152 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-kube-api-access-d6bbq" (OuterVolumeSpecName: "kube-api-access-d6bbq") pod "31f47884-061b-4eb7-b27b-91d855d9bf3a" (UID: "31f47884-061b-4eb7-b27b-91d855d9bf3a"). InnerVolumeSpecName "kube-api-access-d6bbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:03:44.568283 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.568257 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31f47884-061b-4eb7-b27b-91d855d9bf3a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "31f47884-061b-4eb7-b27b-91d855d9bf3a" (UID: "31f47884-061b-4eb7-b27b-91d855d9bf3a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:03:44.660455 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.660414 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/31f47884-061b-4eb7-b27b-91d855d9bf3a-trusted-ca\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:03:44.660455 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.660446 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-registry-tls\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:03:44.660455 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.660458 2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-bound-sa-token\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:03:44.660455 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.660467 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6bbq\" (UniqueName: \"kubernetes.io/projected/31f47884-061b-4eb7-b27b-91d855d9bf3a-kube-api-access-d6bbq\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:03:44.660717 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.660476 2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/31f47884-061b-4eb7-b27b-91d855d9bf3a-ca-trust-extracted\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:03:44.660717 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.660486 2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/31f47884-061b-4eb7-b27b-91d855d9bf3a-image-registry-private-configuration\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:03:44.660717 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:44.660495 2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/31f47884-061b-4eb7-b27b-91d855d9bf3a-installation-pull-secrets\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:03:45.152901 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:45.152869 2576 generic.go:356] "Generic (PLEG): container finished" podID="31f47884-061b-4eb7-b27b-91d855d9bf3a" containerID="1267fd43037e4ef56559407234c946bf3b17f2730e50a965ea4b99e04951e8c8" exitCode=0 Sep 10 22:03:45.153097 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:45.152921 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" Sep 10 22:03:45.153097 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:45.152959 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" event={"ID":"31f47884-061b-4eb7-b27b-91d855d9bf3a","Type":"ContainerDied","Data":"1267fd43037e4ef56559407234c946bf3b17f2730e50a965ea4b99e04951e8c8"} Sep 10 22:03:45.153097 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:45.153020 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cdb65cf65-bnlmv" event={"ID":"31f47884-061b-4eb7-b27b-91d855d9bf3a","Type":"ContainerDied","Data":"80ab84a2c2e3598507f8e4a3b7c2b75d38542a2345ad4ed8e6d6d633443e03c7"} Sep 10 22:03:45.153097 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:45.153061 2576 scope.go:117] "RemoveContainer" containerID="1267fd43037e4ef56559407234c946bf3b17f2730e50a965ea4b99e04951e8c8" Sep 10 22:03:45.162118 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:45.162089 2576 scope.go:117] "RemoveContainer" containerID="1267fd43037e4ef56559407234c946bf3b17f2730e50a965ea4b99e04951e8c8" Sep 10 22:03:45.162421 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:03:45.162401 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1267fd43037e4ef56559407234c946bf3b17f2730e50a965ea4b99e04951e8c8\": container with ID starting with 1267fd43037e4ef56559407234c946bf3b17f2730e50a965ea4b99e04951e8c8 not found: ID does not exist" containerID="1267fd43037e4ef56559407234c946bf3b17f2730e50a965ea4b99e04951e8c8" Sep 10 22:03:45.162479 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:45.162431 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1267fd43037e4ef56559407234c946bf3b17f2730e50a965ea4b99e04951e8c8"} err="failed to get container status \"1267fd43037e4ef56559407234c946bf3b17f2730e50a965ea4b99e04951e8c8\": rpc error: code = NotFound desc = could not find container \"1267fd43037e4ef56559407234c946bf3b17f2730e50a965ea4b99e04951e8c8\": container with ID starting with 1267fd43037e4ef56559407234c946bf3b17f2730e50a965ea4b99e04951e8c8 not found: ID does not exist" Sep 10 22:03:45.173248 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:45.173217 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cdb65cf65-bnlmv"] Sep 10 22:03:45.178849 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:45.178820 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6cdb65cf65-bnlmv"] Sep 10 22:03:45.512641 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:45.512561 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:45.563580 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:45.563545 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:46.430587 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:46.430556 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31f47884-061b-4eb7-b27b-91d855d9bf3a" path="/var/lib/kubelet/pods/31f47884-061b-4eb7-b27b-91d855d9bf3a/volumes" Sep 10 22:03:47.164744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:47.164707 2576 generic.go:356] "Generic (PLEG): container finished" podID="1566f5ee-e336-455a-b28c-a77e91708be2" containerID="44da26366d214d738fbc9cf04413652aafd5bf361ad86de423ec25584d9fd625" exitCode=0 Sep 10 22:03:47.165244 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:47.164791 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" event={"ID":"1566f5ee-e336-455a-b28c-a77e91708be2","Type":"ContainerDied","Data":"44da26366d214d738fbc9cf04413652aafd5bf361ad86de423ec25584d9fd625"} Sep 10 22:03:47.165244 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:47.165193 2576 scope.go:117] "RemoveContainer" containerID="44da26366d214d738fbc9cf04413652aafd5bf361ad86de423ec25584d9fd625" Sep 10 22:03:48.170308 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:48.170272 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-2t7c5" event={"ID":"1566f5ee-e336-455a-b28c-a77e91708be2","Type":"ContainerStarted","Data":"c6a7b1b794b371b6c51cef822fba0ea18fae9dd52b6cb68f23e4d72418e0673d"} Sep 10 22:03:48.529454 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:48.529357 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:48.591577 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:48.591544 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:49.089571 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:49.089543 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:49.156513 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:49.156478 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:50.675034 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:50.675004 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:50.727590 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:50.727556 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:55.484940 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:55.484903 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:56.877927 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:56.877886 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:56.965782 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:56.965740 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:57.143916 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:57.143830 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:03:57.217967 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:03:57.217935 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:04:18.835997 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:18.835955 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:18.855619 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:18.855592 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:19.308181 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:19.308103 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:31.535246 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:31.535199 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:04:31.852499 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:31.852461 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:04:32.598433 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:32.598387 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:04:32.598943 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:32.598698 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="alertmanager" containerID="cri-o://e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073" gracePeriod=120 Sep 10 22:04:32.598943 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:32.598753 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="kube-rbac-proxy-metric" containerID="cri-o://960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a" gracePeriod=120 Sep 10 22:04:32.598943 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:32.598841 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="prom-label-proxy" containerID="cri-o://5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a" gracePeriod=120 Sep 10 22:04:32.598943 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:32.598869 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="kube-rbac-proxy" containerID="cri-o://011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7" gracePeriod=120 Sep 10 22:04:32.598943 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:32.598873 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="kube-rbac-proxy-web" containerID="cri-o://c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259" gracePeriod=120 Sep 10 22:04:32.599190 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:32.598952 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="config-reloader" containerID="cri-o://3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef" gracePeriod=120 Sep 10 22:04:33.347501 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.347463 2576 generic.go:356] "Generic (PLEG): container finished" podID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerID="5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a" exitCode=0 Sep 10 22:04:33.347501 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.347490 2576 generic.go:356] "Generic (PLEG): container finished" podID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerID="011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7" exitCode=0 Sep 10 22:04:33.347501 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.347496 2576 generic.go:356] "Generic (PLEG): container finished" podID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerID="3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef" exitCode=0 Sep 10 22:04:33.347501 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.347502 2576 generic.go:356] "Generic (PLEG): container finished" podID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerID="e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073" exitCode=0 Sep 10 22:04:33.347860 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.347536 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"024917fc-4a5d-4cd5-9b26-1ec98de340ed","Type":"ContainerDied","Data":"5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a"} Sep 10 22:04:33.347860 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.347648 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"024917fc-4a5d-4cd5-9b26-1ec98de340ed","Type":"ContainerDied","Data":"011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7"} Sep 10 22:04:33.347860 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.347690 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"024917fc-4a5d-4cd5-9b26-1ec98de340ed","Type":"ContainerDied","Data":"3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef"} Sep 10 22:04:33.347860 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.347728 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"024917fc-4a5d-4cd5-9b26-1ec98de340ed","Type":"ContainerDied","Data":"e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073"} Sep 10 22:04:33.847687 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.847658 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:33.971079 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.970989 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-web-config\") pod \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " Sep 10 22:04:33.971079 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.971064 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfn8x\" (UniqueName: \"kubernetes.io/projected/024917fc-4a5d-4cd5-9b26-1ec98de340ed-kube-api-access-wfn8x\") pod \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " Sep 10 22:04:33.971307 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.971085 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/024917fc-4a5d-4cd5-9b26-1ec98de340ed-config-out\") pod \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " Sep 10 22:04:33.971307 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.971103 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy-web\") pod \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " Sep 10 22:04:33.971307 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.971132 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/024917fc-4a5d-4cd5-9b26-1ec98de340ed-metrics-client-ca\") pod \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " Sep 10 22:04:33.971307 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.971156 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-cluster-tls-config\") pod \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " Sep 10 22:04:33.971307 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.971188 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/024917fc-4a5d-4cd5-9b26-1ec98de340ed-alertmanager-trusted-ca-bundle\") pod \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " Sep 10 22:04:33.971307 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.971219 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/024917fc-4a5d-4cd5-9b26-1ec98de340ed-alertmanager-main-db\") pod \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " Sep 10 22:04:33.971609 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.971583 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/024917fc-4a5d-4cd5-9b26-1ec98de340ed-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "024917fc-4a5d-4cd5-9b26-1ec98de340ed" (UID: "024917fc-4a5d-4cd5-9b26-1ec98de340ed"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:04:33.971667 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.971607 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/024917fc-4a5d-4cd5-9b26-1ec98de340ed-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "024917fc-4a5d-4cd5-9b26-1ec98de340ed" (UID: "024917fc-4a5d-4cd5-9b26-1ec98de340ed"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:04:33.971720 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.971669 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy\") pod \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " Sep 10 22:04:33.971720 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.971704 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " Sep 10 22:04:33.971835 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.971738 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-main-tls\") pod \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " Sep 10 22:04:33.971835 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.971795 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/024917fc-4a5d-4cd5-9b26-1ec98de340ed-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "024917fc-4a5d-4cd5-9b26-1ec98de340ed" (UID: "024917fc-4a5d-4cd5-9b26-1ec98de340ed"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:04:33.974050 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.974020 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-config-volume\") pod \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " Sep 10 22:04:33.974214 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.974064 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/024917fc-4a5d-4cd5-9b26-1ec98de340ed-tls-assets\") pod \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\" (UID: \"024917fc-4a5d-4cd5-9b26-1ec98de340ed\") " Sep 10 22:04:33.974312 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.974260 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/024917fc-4a5d-4cd5-9b26-1ec98de340ed-config-out" (OuterVolumeSpecName: "config-out") pod "024917fc-4a5d-4cd5-9b26-1ec98de340ed" (UID: "024917fc-4a5d-4cd5-9b26-1ec98de340ed"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:04:33.974457 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.974382 2576 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/024917fc-4a5d-4cd5-9b26-1ec98de340ed-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:33.974457 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.974406 2576 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/024917fc-4a5d-4cd5-9b26-1ec98de340ed-alertmanager-main-db\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:33.974457 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.974421 2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/024917fc-4a5d-4cd5-9b26-1ec98de340ed-config-out\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:33.974457 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.974435 2576 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/024917fc-4a5d-4cd5-9b26-1ec98de340ed-metrics-client-ca\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:33.974457 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.974429 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "024917fc-4a5d-4cd5-9b26-1ec98de340ed" (UID: "024917fc-4a5d-4cd5-9b26-1ec98de340ed"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:33.975286 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.975244 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/024917fc-4a5d-4cd5-9b26-1ec98de340ed-kube-api-access-wfn8x" (OuterVolumeSpecName: "kube-api-access-wfn8x") pod "024917fc-4a5d-4cd5-9b26-1ec98de340ed" (UID: "024917fc-4a5d-4cd5-9b26-1ec98de340ed"). InnerVolumeSpecName "kube-api-access-wfn8x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:04:33.975286 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.975268 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "024917fc-4a5d-4cd5-9b26-1ec98de340ed" (UID: "024917fc-4a5d-4cd5-9b26-1ec98de340ed"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:33.975449 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.975287 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "024917fc-4a5d-4cd5-9b26-1ec98de340ed" (UID: "024917fc-4a5d-4cd5-9b26-1ec98de340ed"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:33.975673 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.975648 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "024917fc-4a5d-4cd5-9b26-1ec98de340ed" (UID: "024917fc-4a5d-4cd5-9b26-1ec98de340ed"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:33.976054 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.976031 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-config-volume" (OuterVolumeSpecName: "config-volume") pod "024917fc-4a5d-4cd5-9b26-1ec98de340ed" (UID: "024917fc-4a5d-4cd5-9b26-1ec98de340ed"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:33.976286 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.976269 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/024917fc-4a5d-4cd5-9b26-1ec98de340ed-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "024917fc-4a5d-4cd5-9b26-1ec98de340ed" (UID: "024917fc-4a5d-4cd5-9b26-1ec98de340ed"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:04:33.978945 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.978914 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "024917fc-4a5d-4cd5-9b26-1ec98de340ed" (UID: "024917fc-4a5d-4cd5-9b26-1ec98de340ed"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:33.985536 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:33.985500 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-web-config" (OuterVolumeSpecName: "web-config") pod "024917fc-4a5d-4cd5-9b26-1ec98de340ed" (UID: "024917fc-4a5d-4cd5-9b26-1ec98de340ed"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:34.075235 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.075182 2576 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-cluster-tls-config\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:34.075235 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.075228 2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:34.075235 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.075243 2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:34.075480 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.075257 2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-main-tls\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:34.075480 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.075270 2576 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:34.075480 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.075281 2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/024917fc-4a5d-4cd5-9b26-1ec98de340ed-tls-assets\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:34.075480 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.075291 2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-web-config\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:34.075480 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.075300 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfn8x\" (UniqueName: \"kubernetes.io/projected/024917fc-4a5d-4cd5-9b26-1ec98de340ed-kube-api-access-wfn8x\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:34.075480 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.075312 2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/024917fc-4a5d-4cd5-9b26-1ec98de340ed-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:34.354889 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.354856 2576 generic.go:356] "Generic (PLEG): container finished" podID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerID="960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a" exitCode=0 Sep 10 22:04:34.354889 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.354880 2576 generic.go:356] "Generic (PLEG): container finished" podID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerID="c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259" exitCode=0 Sep 10 22:04:34.355136 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.354934 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.355136 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.354936 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"024917fc-4a5d-4cd5-9b26-1ec98de340ed","Type":"ContainerDied","Data":"960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a"} Sep 10 22:04:34.355136 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.355020 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"024917fc-4a5d-4cd5-9b26-1ec98de340ed","Type":"ContainerDied","Data":"c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259"} Sep 10 22:04:34.355136 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.355061 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"024917fc-4a5d-4cd5-9b26-1ec98de340ed","Type":"ContainerDied","Data":"d2b164bda1f4e12e181c321eb93279510e330c82ef873a9eab4ad2beb39aa0cc"} Sep 10 22:04:34.355136 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.355088 2576 scope.go:117] "RemoveContainer" containerID="5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a" Sep 10 22:04:34.362980 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.362912 2576 scope.go:117] "RemoveContainer" containerID="960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a" Sep 10 22:04:34.370272 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.370251 2576 scope.go:117] "RemoveContainer" containerID="011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7" Sep 10 22:04:34.377433 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.377406 2576 scope.go:117] "RemoveContainer" containerID="c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259" Sep 10 22:04:34.382568 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.382541 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:04:34.385643 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.385611 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:04:34.386678 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.386659 2576 scope.go:117] "RemoveContainer" containerID="3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef" Sep 10 22:04:34.394326 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.394307 2576 scope.go:117] "RemoveContainer" containerID="e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073" Sep 10 22:04:34.401847 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.401827 2576 scope.go:117] "RemoveContainer" containerID="80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7" Sep 10 22:04:34.409745 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.409725 2576 scope.go:117] "RemoveContainer" containerID="5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a" Sep 10 22:04:34.410089 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:04:34.410065 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a\": container with ID starting with 5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a not found: ID does not exist" containerID="5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a" Sep 10 22:04:34.410140 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.410103 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a"} err="failed to get container status \"5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a\": rpc error: code = NotFound desc = could not find container \"5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a\": container with ID starting with 5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a not found: ID does not exist" Sep 10 22:04:34.410140 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.410123 2576 scope.go:117] "RemoveContainer" containerID="960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a" Sep 10 22:04:34.410367 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:04:34.410351 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a\": container with ID starting with 960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a not found: ID does not exist" containerID="960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a" Sep 10 22:04:34.410409 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.410374 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a"} err="failed to get container status \"960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a\": rpc error: code = NotFound desc = could not find container \"960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a\": container with ID starting with 960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a not found: ID does not exist" Sep 10 22:04:34.410409 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.410390 2576 scope.go:117] "RemoveContainer" containerID="011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7" Sep 10 22:04:34.410591 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:04:34.410574 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7\": container with ID starting with 011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7 not found: ID does not exist" containerID="011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7" Sep 10 22:04:34.410632 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.410597 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7"} err="failed to get container status \"011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7\": rpc error: code = NotFound desc = could not find container \"011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7\": container with ID starting with 011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7 not found: ID does not exist" Sep 10 22:04:34.410632 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.410610 2576 scope.go:117] "RemoveContainer" containerID="c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259" Sep 10 22:04:34.410840 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:04:34.410825 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259\": container with ID starting with c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259 not found: ID does not exist" containerID="c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259" Sep 10 22:04:34.410888 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.410844 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259"} err="failed to get container status \"c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259\": rpc error: code = NotFound desc = could not find container \"c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259\": container with ID starting with c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259 not found: ID does not exist" Sep 10 22:04:34.410888 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.410858 2576 scope.go:117] "RemoveContainer" containerID="3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef" Sep 10 22:04:34.411093 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:04:34.411076 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef\": container with ID starting with 3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef not found: ID does not exist" containerID="3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef" Sep 10 22:04:34.411163 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.411102 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef"} err="failed to get container status \"3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef\": rpc error: code = NotFound desc = could not find container \"3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef\": container with ID starting with 3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef not found: ID does not exist" Sep 10 22:04:34.411163 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.411125 2576 scope.go:117] "RemoveContainer" containerID="e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073" Sep 10 22:04:34.411424 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:04:34.411408 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073\": container with ID starting with e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073 not found: ID does not exist" containerID="e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073" Sep 10 22:04:34.411484 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.411429 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073"} err="failed to get container status \"e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073\": rpc error: code = NotFound desc = could not find container \"e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073\": container with ID starting with e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073 not found: ID does not exist" Sep 10 22:04:34.411484 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.411449 2576 scope.go:117] "RemoveContainer" containerID="80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7" Sep 10 22:04:34.411811 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:04:34.411759 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7\": container with ID starting with 80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7 not found: ID does not exist" containerID="80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7" Sep 10 22:04:34.411811 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.411786 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7"} err="failed to get container status \"80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7\": rpc error: code = NotFound desc = could not find container \"80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7\": container with ID starting with 80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7 not found: ID does not exist" Sep 10 22:04:34.411987 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.411820 2576 scope.go:117] "RemoveContainer" containerID="5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a" Sep 10 22:04:34.412168 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.412120 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a"} err="failed to get container status \"5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a\": rpc error: code = NotFound desc = could not find container \"5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a\": container with ID starting with 5bd2dd5f17f24eb10982bc5360f4d1c4c3b9749d757fe641b9f77a730c6bed7a not found: ID does not exist" Sep 10 22:04:34.412168 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.412148 2576 scope.go:117] "RemoveContainer" containerID="960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a" Sep 10 22:04:34.412532 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.412496 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a"} err="failed to get container status \"960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a\": rpc error: code = NotFound desc = could not find container \"960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a\": container with ID starting with 960d52e646ffd5d6a0b3d36a3c440b39b10518401bb4bbc87419e25daded952a not found: ID does not exist" Sep 10 22:04:34.412532 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.412527 2576 scope.go:117] "RemoveContainer" containerID="011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7" Sep 10 22:04:34.412789 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.412772 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7"} err="failed to get container status \"011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7\": rpc error: code = NotFound desc = could not find container \"011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7\": container with ID starting with 011463cc7127b841702ff96d4841f92573da89ff37016a74fca3f7611c4c89e7 not found: ID does not exist" Sep 10 22:04:34.412789 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.412789 2576 scope.go:117] "RemoveContainer" containerID="c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259" Sep 10 22:04:34.413062 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.413044 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259"} err="failed to get container status \"c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259\": rpc error: code = NotFound desc = could not find container \"c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259\": container with ID starting with c29e404059f83bd71d79994673efcbc78b7275bc41b18257b1221ca340f97259 not found: ID does not exist" Sep 10 22:04:34.413107 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.413063 2576 scope.go:117] "RemoveContainer" containerID="3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef" Sep 10 22:04:34.413299 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.413281 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef"} err="failed to get container status \"3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef\": rpc error: code = NotFound desc = could not find container \"3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef\": container with ID starting with 3b58a9444780d9b9381680bd2d4b7b2a43beed130d1adeecf53901dfd033ceef not found: ID does not exist" Sep 10 22:04:34.413374 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.413301 2576 scope.go:117] "RemoveContainer" containerID="e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073" Sep 10 22:04:34.413564 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.413548 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073"} err="failed to get container status \"e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073\": rpc error: code = NotFound desc = could not find container \"e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073\": container with ID starting with e1235e469153e32474bcbc70f1eeda82c17132866f14eb6c627389f86ba2d073 not found: ID does not exist" Sep 10 22:04:34.413611 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.413564 2576 scope.go:117] "RemoveContainer" containerID="80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7" Sep 10 22:04:34.413816 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.413778 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7"} err="failed to get container status \"80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7\": rpc error: code = NotFound desc = could not find container \"80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7\": container with ID starting with 80dec95d8a9eb0608911f29058702a71a5a565e9796b9a2fbc32c35d8f6faac7 not found: ID does not exist" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.431648 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" path="/var/lib/kubelet/pods/024917fc-4a5d-4cd5-9b26-1ec98de340ed/volumes" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432362 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432725 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="config-reloader" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432738 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="config-reloader" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432753 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="alertmanager" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432761 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="alertmanager" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432770 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="kube-rbac-proxy-metric" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432778 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="kube-rbac-proxy-metric" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432816 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31f47884-061b-4eb7-b27b-91d855d9bf3a" containerName="registry" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432825 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="31f47884-061b-4eb7-b27b-91d855d9bf3a" containerName="registry" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432836 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="init-config-reloader" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432844 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="init-config-reloader" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432856 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="prom-label-proxy" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432864 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="prom-label-proxy" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432876 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="kube-rbac-proxy" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432884 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="kube-rbac-proxy" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432899 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="kube-rbac-proxy-web" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432907 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="kube-rbac-proxy-web" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432971 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="alertmanager" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432980 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="kube-rbac-proxy-metric" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432989 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="31f47884-061b-4eb7-b27b-91d855d9bf3a" containerName="registry" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.432997 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="config-reloader" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.433008 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="kube-rbac-proxy" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.433017 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="kube-rbac-proxy-web" Sep 10 22:04:34.433570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.433025 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="024917fc-4a5d-4cd5-9b26-1ec98de340ed" containerName="prom-label-proxy" Sep 10 22:04:34.437157 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.437133 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.440820 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.440773 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 22:04:34.440954 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.440856 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 22:04:34.440954 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.440893 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 22:04:34.440954 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.440893 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 22:04:34.440954 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.440899 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-v2jpn\"" Sep 10 22:04:34.441200 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.441184 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 22:04:34.441276 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.441197 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 22:04:34.441276 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.441212 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 22:04:34.441439 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.441412 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 22:04:34.446344 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.446325 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 22:04:34.454097 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.454068 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:04:34.478324 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.478249 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtqs2\" (UniqueName: \"kubernetes.io/projected/3ed32fc6-5af6-422b-abcd-0639485ebc5d-kube-api-access-xtqs2\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.478520 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.478378 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ed32fc6-5af6-422b-abcd-0639485ebc5d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.478520 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.478427 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.478520 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.478454 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3ed32fc6-5af6-422b-abcd-0639485ebc5d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.478738 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.478585 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.478738 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.478657 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ed32fc6-5af6-422b-abcd-0639485ebc5d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.478738 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.478694 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-web-config\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.478916 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.478774 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.478916 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.478837 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.478916 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.478870 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-config-volume\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.478916 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.478894 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.479367 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.478919 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ed32fc6-5af6-422b-abcd-0639485ebc5d-config-out\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.479367 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.478956 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ed32fc6-5af6-422b-abcd-0639485ebc5d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.580083 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.580036 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ed32fc6-5af6-422b-abcd-0639485ebc5d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.580083 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.580082 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtqs2\" (UniqueName: \"kubernetes.io/projected/3ed32fc6-5af6-422b-abcd-0639485ebc5d-kube-api-access-xtqs2\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.580319 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.580180 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ed32fc6-5af6-422b-abcd-0639485ebc5d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.580319 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.580206 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.580319 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.580224 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3ed32fc6-5af6-422b-abcd-0639485ebc5d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.580319 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.580247 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.580319 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.580283 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ed32fc6-5af6-422b-abcd-0639485ebc5d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.580319 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.580318 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-web-config\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.580659 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.580359 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.580659 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.580385 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.580659 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.580414 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-config-volume\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.580659 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.580434 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.580659 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.580456 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ed32fc6-5af6-422b-abcd-0639485ebc5d-config-out\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.580659 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.580648 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3ed32fc6-5af6-422b-abcd-0639485ebc5d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.580996 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.580881 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ed32fc6-5af6-422b-abcd-0639485ebc5d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.581691 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.581662 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ed32fc6-5af6-422b-abcd-0639485ebc5d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.583463 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.583410 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.583682 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.583576 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.583932 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.583914 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ed32fc6-5af6-422b-abcd-0639485ebc5d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.584058 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.584038 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.584107 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.584044 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-web-config\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.584107 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.584085 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.584289 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.584261 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ed32fc6-5af6-422b-abcd-0639485ebc5d-config-out\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.584324 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.584304 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-config-volume\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.585075 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.585054 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ed32fc6-5af6-422b-abcd-0639485ebc5d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.588185 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.588164 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtqs2\" (UniqueName: \"kubernetes.io/projected/3ed32fc6-5af6-422b-abcd-0639485ebc5d-kube-api-access-xtqs2\") pod \"alertmanager-main-0\" (UID: \"3ed32fc6-5af6-422b-abcd-0639485ebc5d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:34.747989 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:34.747952 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:04:35.027094 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:35.027016 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:04:35.029858 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:04:35.029826 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ed32fc6_5af6_422b_abcd_0639485ebc5d.slice/crio-f59b280bc412bfb4e5e98bc634b807a802bac9e3b4459c3aa96b54d1d7ea923d WatchSource:0}: Error finding container f59b280bc412bfb4e5e98bc634b807a802bac9e3b4459c3aa96b54d1d7ea923d: Status 404 returned error can't find the container with id f59b280bc412bfb4e5e98bc634b807a802bac9e3b4459c3aa96b54d1d7ea923d Sep 10 22:04:35.360985 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:35.360947 2576 generic.go:356] "Generic (PLEG): container finished" podID="3ed32fc6-5af6-422b-abcd-0639485ebc5d" containerID="4e1479870067a23c4937220a95ef2f73f06e0cd55758affd7e03a3861d6614ca" exitCode=0 Sep 10 22:04:35.361132 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:35.361017 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed32fc6-5af6-422b-abcd-0639485ebc5d","Type":"ContainerDied","Data":"4e1479870067a23c4937220a95ef2f73f06e0cd55758affd7e03a3861d6614ca"} Sep 10 22:04:35.361132 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:35.361098 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed32fc6-5af6-422b-abcd-0639485ebc5d","Type":"ContainerStarted","Data":"f59b280bc412bfb4e5e98bc634b807a802bac9e3b4459c3aa96b54d1d7ea923d"} Sep 10 22:04:36.368977 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.368941 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed32fc6-5af6-422b-abcd-0639485ebc5d","Type":"ContainerStarted","Data":"8c92d787fc767b81ec73745e5b6f1273aa4fbdfc3d6f02c448da8eceb2028239"} Sep 10 22:04:36.369361 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.369010 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed32fc6-5af6-422b-abcd-0639485ebc5d","Type":"ContainerStarted","Data":"271adc0949dbae3b3627b854a6e4fe0a9f9a1c9800d08ebee317deb12acb79dd"} Sep 10 22:04:36.369361 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.369038 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed32fc6-5af6-422b-abcd-0639485ebc5d","Type":"ContainerStarted","Data":"3e8a6153a63541f32e1c0220ba13fffef419cd22f904643d50c8e08bfdc4da71"} Sep 10 22:04:36.369361 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.369063 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed32fc6-5af6-422b-abcd-0639485ebc5d","Type":"ContainerStarted","Data":"c9ac45d72a098eb666b3995815aee2fa1305fb0c76bb1e4165d56d1870a74869"} Sep 10 22:04:36.369361 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.369089 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed32fc6-5af6-422b-abcd-0639485ebc5d","Type":"ContainerStarted","Data":"48dd1bd812b11d7fde24816230cc9980e491a8e8549290068d331b821e5906b9"} Sep 10 22:04:36.369361 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.369111 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3ed32fc6-5af6-422b-abcd-0639485ebc5d","Type":"ContainerStarted","Data":"9c3e3a0c4cca1cd105a21e8a03a89f470376617c6346b81079e68c7d2e904487"} Sep 10 22:04:36.397165 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.397095 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.383381633 podStartE2EDuration="2.397076727s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.695094ms" isStatefulPod=false podCreationTimestamp="2026-09-10 22:04:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:04:36.3955861 +0000 UTC m=+206.517284882" watchObservedRunningTime="2026-09-10 22:04:36.397076727 +0000 UTC m=+206.518775509" Sep 10 22:04:36.674128 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.674011 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6c44fd675d-t98kh"] Sep 10 22:04:36.679398 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.679371 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.683464 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.683438 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 10 22:04:36.683704 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.683686 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 10 22:04:36.684921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.684896 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 10 22:04:36.685061 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.684963 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 10 22:04:36.685061 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.684995 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 10 22:04:36.685492 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.685469 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-tl8pc\"" Sep 10 22:04:36.698364 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.698309 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 10 22:04:36.701454 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.701425 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c44fd675d-t98kh"] Sep 10 22:04:36.799773 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.799733 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-serving-certs-ca-bundle\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.799773 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.799773 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-secret-telemeter-client\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.800084 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.799791 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-telemeter-client-tls\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.800084 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.799836 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.800084 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.799918 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvdcv\" (UniqueName: \"kubernetes.io/projected/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-kube-api-access-cvdcv\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.800084 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.799997 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-federate-client-tls\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.800084 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.800042 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.800239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.800105 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-metrics-client-ca\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.901554 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.901520 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-serving-certs-ca-bundle\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.901554 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.901554 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-secret-telemeter-client\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.901881 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.901578 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-telemeter-client-tls\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.901881 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.901623 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.901881 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.901668 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvdcv\" (UniqueName: \"kubernetes.io/projected/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-kube-api-access-cvdcv\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.901881 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.901697 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-federate-client-tls\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.901881 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.901721 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.901881 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.901756 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-metrics-client-ca\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.902462 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.902430 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-serving-certs-ca-bundle\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.902588 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.902440 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-metrics-client-ca\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.902867 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.902841 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.904443 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.904418 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-federate-client-tls\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.904547 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.904426 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.904765 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.904743 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-telemeter-client-tls\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.904765 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.904758 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-secret-telemeter-client\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.910781 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.910751 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvdcv\" (UniqueName: \"kubernetes.io/projected/ff35ebdf-3271-4d7b-8490-dd9c25d9e714-kube-api-access-cvdcv\") pod \"telemeter-client-6c44fd675d-t98kh\" (UID: \"ff35ebdf-3271-4d7b-8490-dd9c25d9e714\") " pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:36.982888 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.982774 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 22:04:36.983792 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.983556 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227" gracePeriod=600 Sep 10 22:04:36.983792 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.983623 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="thanos-sidecar" containerID="cri-o://2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff" gracePeriod=600 Sep 10 22:04:36.983792 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.983673 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="kube-rbac-proxy" containerID="cri-o://8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245" gracePeriod=600 Sep 10 22:04:36.983792 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.983573 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="prometheus" containerID="cri-o://858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990" gracePeriod=600 Sep 10 22:04:36.983792 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.983642 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="config-reloader" containerID="cri-o://12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1" gracePeriod=600 Sep 10 22:04:36.983792 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.983635 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="kube-rbac-proxy-web" containerID="cri-o://b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c" gracePeriod=600 Sep 10 22:04:36.992218 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:36.989922 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" Sep 10 22:04:37.242043 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.241972 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.292327 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.292227 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c44fd675d-t98kh"] Sep 10 22:04:37.296669 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:04:37.296633 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff35ebdf_3271_4d7b_8490_dd9c25d9e714.slice/crio-dc99d0e3ddbf4da08b94c008c62456436b86099f72b2582b2a26536dd7e09fa8 WatchSource:0}: Error finding container dc99d0e3ddbf4da08b94c008c62456436b86099f72b2582b2a26536dd7e09fa8: Status 404 returned error can't find the container with id dc99d0e3ddbf4da08b94c008c62456436b86099f72b2582b2a26536dd7e09fa8 Sep 10 22:04:37.305586 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.305557 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-tls\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.305705 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.305596 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-kube-rbac-proxy\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.305705 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.305642 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-1\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.305705 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.305667 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-0\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.305705 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.305701 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-db\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.305993 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.305960 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-thanos-prometheus-http-client-file\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.306057 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306043 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-2\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.306125 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306080 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f13be60c-9156-4e4a-a067-7764705e6564-config-out\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.306125 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306111 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.306222 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306114 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:04:37.306222 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306141 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-serving-certs-ca-bundle\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.306222 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306172 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.306222 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306205 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-metrics-client-ca\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.306422 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306231 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f13be60c-9156-4e4a-a067-7764705e6564-tls-assets\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.306422 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306285 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-config\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.306422 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306310 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-metrics-client-certs\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.306422 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306338 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-kubelet-serving-ca-bundle\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.306422 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306367 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-grpc-tls\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.306422 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306405 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htmql\" (UniqueName: \"kubernetes.io/projected/f13be60c-9156-4e4a-a067-7764705e6564-kube-api-access-htmql\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.306721 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306434 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-web-config\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.306721 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306468 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:04:37.306721 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306488 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-trusted-ca-bundle\") pod \"f13be60c-9156-4e4a-a067-7764705e6564\" (UID: \"f13be60c-9156-4e4a-a067-7764705e6564\") " Sep 10 22:04:37.306923 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306761 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:04:37.306923 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306818 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.306923 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.306839 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.308353 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.307360 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:04:37.308353 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.307421 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:04:37.308353 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.307428 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:04:37.308353 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.308115 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:04:37.308353 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.308293 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:04:37.310157 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.310037 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:37.310157 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.310120 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:37.310734 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.310678 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:37.310853 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.310782 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-config" (OuterVolumeSpecName: "config") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:37.311123 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.311100 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13be60c-9156-4e4a-a067-7764705e6564-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:04:37.311559 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.311526 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13be60c-9156-4e4a-a067-7764705e6564-config-out" (OuterVolumeSpecName: "config-out") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:04:37.311697 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.311676 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13be60c-9156-4e4a-a067-7764705e6564-kube-api-access-htmql" (OuterVolumeSpecName: "kube-api-access-htmql") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "kube-api-access-htmql". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:04:37.312287 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.312251 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:37.312386 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.312349 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:37.312675 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.312645 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:37.312784 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.312764 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:37.326884 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.326844 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-web-config" (OuterVolumeSpecName: "web-config") pod "f13be60c-9156-4e4a-a067-7764705e6564" (UID: "f13be60c-9156-4e4a-a067-7764705e6564"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:04:37.376055 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.376025 2576 generic.go:356] "Generic (PLEG): container finished" podID="f13be60c-9156-4e4a-a067-7764705e6564" containerID="7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227" exitCode=0 Sep 10 22:04:37.376055 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.376049 2576 generic.go:356] "Generic (PLEG): container finished" podID="f13be60c-9156-4e4a-a067-7764705e6564" containerID="8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245" exitCode=0 Sep 10 22:04:37.376055 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.376055 2576 generic.go:356] "Generic (PLEG): container finished" podID="f13be60c-9156-4e4a-a067-7764705e6564" containerID="b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c" exitCode=0 Sep 10 22:04:37.376055 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.376061 2576 generic.go:356] "Generic (PLEG): container finished" podID="f13be60c-9156-4e4a-a067-7764705e6564" containerID="2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff" exitCode=0 Sep 10 22:04:37.376662 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.376066 2576 generic.go:356] "Generic (PLEG): container finished" podID="f13be60c-9156-4e4a-a067-7764705e6564" containerID="12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1" exitCode=0 Sep 10 22:04:37.376662 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.376074 2576 generic.go:356] "Generic (PLEG): container finished" podID="f13be60c-9156-4e4a-a067-7764705e6564" containerID="858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990" exitCode=0 Sep 10 22:04:37.376662 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.376110 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.376662 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.376110 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f13be60c-9156-4e4a-a067-7764705e6564","Type":"ContainerDied","Data":"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227"} Sep 10 22:04:37.376662 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.376194 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f13be60c-9156-4e4a-a067-7764705e6564","Type":"ContainerDied","Data":"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245"} Sep 10 22:04:37.376662 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.376236 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f13be60c-9156-4e4a-a067-7764705e6564","Type":"ContainerDied","Data":"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c"} Sep 10 22:04:37.376662 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.376254 2576 scope.go:117] "RemoveContainer" containerID="7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227" Sep 10 22:04:37.376662 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.376274 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f13be60c-9156-4e4a-a067-7764705e6564","Type":"ContainerDied","Data":"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff"} Sep 10 22:04:37.376662 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.376306 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f13be60c-9156-4e4a-a067-7764705e6564","Type":"ContainerDied","Data":"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1"} Sep 10 22:04:37.376662 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.376343 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f13be60c-9156-4e4a-a067-7764705e6564","Type":"ContainerDied","Data":"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990"} Sep 10 22:04:37.376662 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.376373 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f13be60c-9156-4e4a-a067-7764705e6564","Type":"ContainerDied","Data":"5dac3eaf47feb38bc6035a2232ff9b771beadd9584bd014ed31323b4dbfab96a"} Sep 10 22:04:37.377583 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.377549 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" event={"ID":"ff35ebdf-3271-4d7b-8490-dd9c25d9e714","Type":"ContainerStarted","Data":"dc99d0e3ddbf4da08b94c008c62456436b86099f72b2582b2a26536dd7e09fa8"} Sep 10 22:04:37.385581 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.385364 2576 scope.go:117] "RemoveContainer" containerID="8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245" Sep 10 22:04:37.393996 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.393976 2576 scope.go:117] "RemoveContainer" containerID="b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c" Sep 10 22:04:37.401303 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.401270 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 22:04:37.402229 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.402216 2576 scope.go:117] "RemoveContainer" containerID="2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff" Sep 10 22:04:37.405857 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.405832 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 22:04:37.407220 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407197 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407225 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-k8s-db\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407241 2576 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407254 2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f13be60c-9156-4e4a-a067-7764705e6564-config-out\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407267 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407281 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407296 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407311 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-metrics-client-ca\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407323 2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f13be60c-9156-4e4a-a067-7764705e6564-tls-assets\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407337 2576 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-config\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407653 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407349 2576 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-metrics-client-certs\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407653 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407362 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407653 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407376 2576 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-grpc-tls\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407653 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407390 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htmql\" (UniqueName: \"kubernetes.io/projected/f13be60c-9156-4e4a-a067-7764705e6564-kube-api-access-htmql\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407653 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407403 2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-web-config\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407653 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407415 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13be60c-9156-4e4a-a067-7764705e6564-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407653 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407430 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.407653 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.407444 2576 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f13be60c-9156-4e4a-a067-7764705e6564-secret-kube-rbac-proxy\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:04:37.410230 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.410215 2576 scope.go:117] "RemoveContainer" containerID="12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1" Sep 10 22:04:37.417434 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.417402 2576 scope.go:117] "RemoveContainer" containerID="858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990" Sep 10 22:04:37.425438 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.425409 2576 scope.go:117] "RemoveContainer" containerID="fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815" Sep 10 22:04:37.432788 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.432768 2576 scope.go:117] "RemoveContainer" containerID="7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227" Sep 10 22:04:37.433142 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:04:37.433124 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227\": container with ID starting with 7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227 not found: ID does not exist" containerID="7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227" Sep 10 22:04:37.433205 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.433152 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227"} err="failed to get container status \"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227\": rpc error: code = NotFound desc = could not find container \"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227\": container with ID starting with 7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227 not found: ID does not exist" Sep 10 22:04:37.433205 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.433169 2576 scope.go:117] "RemoveContainer" containerID="8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245" Sep 10 22:04:37.433390 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:04:37.433373 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245\": container with ID starting with 8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245 not found: ID does not exist" containerID="8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245" Sep 10 22:04:37.433440 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.433393 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245"} err="failed to get container status \"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245\": rpc error: code = NotFound desc = could not find container \"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245\": container with ID starting with 8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245 not found: ID does not exist" Sep 10 22:04:37.433440 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.433407 2576 scope.go:117] "RemoveContainer" containerID="b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c" Sep 10 22:04:37.433658 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:04:37.433640 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c\": container with ID starting with b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c not found: ID does not exist" containerID="b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c" Sep 10 22:04:37.433769 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.433665 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c"} err="failed to get container status \"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c\": rpc error: code = NotFound desc = could not find container \"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c\": container with ID starting with b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c not found: ID does not exist" Sep 10 22:04:37.433769 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.433684 2576 scope.go:117] "RemoveContainer" containerID="2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff" Sep 10 22:04:37.434181 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:04:37.434077 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff\": container with ID starting with 2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff not found: ID does not exist" containerID="2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff" Sep 10 22:04:37.434181 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.434112 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff"} err="failed to get container status \"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff\": rpc error: code = NotFound desc = could not find container \"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff\": container with ID starting with 2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff not found: ID does not exist" Sep 10 22:04:37.434371 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.434335 2576 scope.go:117] "RemoveContainer" containerID="12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1" Sep 10 22:04:37.434781 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:04:37.434727 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1\": container with ID starting with 12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1 not found: ID does not exist" containerID="12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1" Sep 10 22:04:37.434781 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.434759 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1"} err="failed to get container status \"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1\": rpc error: code = NotFound desc = could not find container \"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1\": container with ID starting with 12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1 not found: ID does not exist" Sep 10 22:04:37.434989 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.434783 2576 scope.go:117] "RemoveContainer" containerID="858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990" Sep 10 22:04:37.435173 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:04:37.435149 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990\": container with ID starting with 858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990 not found: ID does not exist" containerID="858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990" Sep 10 22:04:37.435247 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.435177 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990"} err="failed to get container status \"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990\": rpc error: code = NotFound desc = could not find container \"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990\": container with ID starting with 858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990 not found: ID does not exist" Sep 10 22:04:37.435247 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.435190 2576 scope.go:117] "RemoveContainer" containerID="fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815" Sep 10 22:04:37.435460 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:04:37.435441 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815\": container with ID starting with fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815 not found: ID does not exist" containerID="fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815" Sep 10 22:04:37.435515 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.435464 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815"} err="failed to get container status \"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815\": rpc error: code = NotFound desc = could not find container \"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815\": container with ID starting with fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815 not found: ID does not exist" Sep 10 22:04:37.435515 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.435480 2576 scope.go:117] "RemoveContainer" containerID="7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227" Sep 10 22:04:37.435734 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.435715 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227"} err="failed to get container status \"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227\": rpc error: code = NotFound desc = could not find container \"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227\": container with ID starting with 7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227 not found: ID does not exist" Sep 10 22:04:37.435820 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.435733 2576 scope.go:117] "RemoveContainer" containerID="8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245" Sep 10 22:04:37.436018 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.435997 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245"} err="failed to get container status \"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245\": rpc error: code = NotFound desc = could not find container \"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245\": container with ID starting with 8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245 not found: ID does not exist" Sep 10 22:04:37.436082 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.436017 2576 scope.go:117] "RemoveContainer" containerID="b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c" Sep 10 22:04:37.436242 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.436224 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c"} err="failed to get container status \"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c\": rpc error: code = NotFound desc = could not find container \"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c\": container with ID starting with b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c not found: ID does not exist" Sep 10 22:04:37.436302 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.436244 2576 scope.go:117] "RemoveContainer" containerID="2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff" Sep 10 22:04:37.436507 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.436486 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff"} err="failed to get container status \"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff\": rpc error: code = NotFound desc = could not find container \"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff\": container with ID starting with 2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff not found: ID does not exist" Sep 10 22:04:37.436566 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.436507 2576 scope.go:117] "RemoveContainer" containerID="12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1" Sep 10 22:04:37.436614 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.436576 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 22:04:37.436749 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.436731 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1"} err="failed to get container status \"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1\": rpc error: code = NotFound desc = could not find container \"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1\": container with ID starting with 12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1 not found: ID does not exist" Sep 10 22:04:37.436831 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.436750 2576 scope.go:117] "RemoveContainer" containerID="858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990" Sep 10 22:04:37.436984 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.436963 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990"} err="failed to get container status \"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990\": rpc error: code = NotFound desc = could not find container \"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990\": container with ID starting with 858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990 not found: ID does not exist" Sep 10 22:04:37.436984 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.436983 2576 scope.go:117] "RemoveContainer" containerID="fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815" Sep 10 22:04:37.437137 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437040 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="config-reloader" Sep 10 22:04:37.437137 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437054 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="config-reloader" Sep 10 22:04:37.437137 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437065 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="prometheus" Sep 10 22:04:37.437137 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437075 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="prometheus" Sep 10 22:04:37.437137 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437094 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="init-config-reloader" Sep 10 22:04:37.437137 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437104 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="init-config-reloader" Sep 10 22:04:37.437137 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437114 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="kube-rbac-proxy" Sep 10 22:04:37.437137 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437119 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="kube-rbac-proxy" Sep 10 22:04:37.437137 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437129 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="kube-rbac-proxy-web" Sep 10 22:04:37.437137 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437137 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="kube-rbac-proxy-web" Sep 10 22:04:37.437505 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437151 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="kube-rbac-proxy-thanos" Sep 10 22:04:37.437505 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437161 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="kube-rbac-proxy-thanos" Sep 10 22:04:37.437505 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437172 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="thanos-sidecar" Sep 10 22:04:37.437505 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437179 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="thanos-sidecar" Sep 10 22:04:37.437505 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437238 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="kube-rbac-proxy-web" Sep 10 22:04:37.437505 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437247 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="prometheus" Sep 10 22:04:37.437505 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437247 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815"} err="failed to get container status \"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815\": rpc error: code = NotFound desc = could not find container \"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815\": container with ID starting with fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815 not found: ID does not exist" Sep 10 22:04:37.437505 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437271 2576 scope.go:117] "RemoveContainer" containerID="7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227" Sep 10 22:04:37.437505 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437256 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="kube-rbac-proxy" Sep 10 22:04:37.437505 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437329 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="config-reloader" Sep 10 22:04:37.437505 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437336 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="kube-rbac-proxy-thanos" Sep 10 22:04:37.437505 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437345 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13be60c-9156-4e4a-a067-7764705e6564" containerName="thanos-sidecar" Sep 10 22:04:37.438019 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437536 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227"} err="failed to get container status \"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227\": rpc error: code = NotFound desc = could not find container \"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227\": container with ID starting with 7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227 not found: ID does not exist" Sep 10 22:04:37.438019 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.437556 2576 scope.go:117] "RemoveContainer" containerID="8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245" Sep 10 22:04:37.438118 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.438075 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245"} err="failed to get container status \"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245\": rpc error: code = NotFound desc = could not find container \"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245\": container with ID starting with 8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245 not found: ID does not exist" Sep 10 22:04:37.438118 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.438099 2576 scope.go:117] "RemoveContainer" containerID="b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c" Sep 10 22:04:37.438381 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.438358 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c"} err="failed to get container status \"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c\": rpc error: code = NotFound desc = could not find container \"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c\": container with ID starting with b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c not found: ID does not exist" Sep 10 22:04:37.438381 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.438380 2576 scope.go:117] "RemoveContainer" containerID="2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff" Sep 10 22:04:37.438660 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.438638 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff"} err="failed to get container status \"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff\": rpc error: code = NotFound desc = could not find container \"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff\": container with ID starting with 2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff not found: ID does not exist" Sep 10 22:04:37.438731 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.438662 2576 scope.go:117] "RemoveContainer" containerID="12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1" Sep 10 22:04:37.438947 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.438927 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1"} err="failed to get container status \"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1\": rpc error: code = NotFound desc = could not find container \"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1\": container with ID starting with 12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1 not found: ID does not exist" Sep 10 22:04:37.438947 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.438946 2576 scope.go:117] "RemoveContainer" containerID="858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990" Sep 10 22:04:37.439225 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.439208 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990"} err="failed to get container status \"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990\": rpc error: code = NotFound desc = could not find container \"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990\": container with ID starting with 858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990 not found: ID does not exist" Sep 10 22:04:37.439225 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.439224 2576 scope.go:117] "RemoveContainer" containerID="fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815" Sep 10 22:04:37.439448 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.439433 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815"} err="failed to get container status \"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815\": rpc error: code = NotFound desc = could not find container \"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815\": container with ID starting with fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815 not found: ID does not exist" Sep 10 22:04:37.439496 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.439450 2576 scope.go:117] "RemoveContainer" containerID="7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227" Sep 10 22:04:37.439644 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.439628 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227"} err="failed to get container status \"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227\": rpc error: code = NotFound desc = could not find container \"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227\": container with ID starting with 7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227 not found: ID does not exist" Sep 10 22:04:37.439687 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.439644 2576 scope.go:117] "RemoveContainer" containerID="8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245" Sep 10 22:04:37.439847 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.439828 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245"} err="failed to get container status \"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245\": rpc error: code = NotFound desc = could not find container \"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245\": container with ID starting with 8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245 not found: ID does not exist" Sep 10 22:04:37.439995 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.439848 2576 scope.go:117] "RemoveContainer" containerID="b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c" Sep 10 22:04:37.440072 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.440054 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c"} err="failed to get container status \"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c\": rpc error: code = NotFound desc = could not find container \"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c\": container with ID starting with b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c not found: ID does not exist" Sep 10 22:04:37.440113 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.440073 2576 scope.go:117] "RemoveContainer" containerID="2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff" Sep 10 22:04:37.440335 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.440319 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff"} err="failed to get container status \"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff\": rpc error: code = NotFound desc = could not find container \"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff\": container with ID starting with 2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff not found: ID does not exist" Sep 10 22:04:37.440335 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.440335 2576 scope.go:117] "RemoveContainer" containerID="12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1" Sep 10 22:04:37.440560 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.440532 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1"} err="failed to get container status \"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1\": rpc error: code = NotFound desc = could not find container \"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1\": container with ID starting with 12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1 not found: ID does not exist" Sep 10 22:04:37.440602 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.440560 2576 scope.go:117] "RemoveContainer" containerID="858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990" Sep 10 22:04:37.440765 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.440750 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990"} err="failed to get container status \"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990\": rpc error: code = NotFound desc = could not find container \"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990\": container with ID starting with 858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990 not found: ID does not exist" Sep 10 22:04:37.440765 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.440764 2576 scope.go:117] "RemoveContainer" containerID="fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815" Sep 10 22:04:37.440997 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.440982 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815"} err="failed to get container status \"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815\": rpc error: code = NotFound desc = could not find container \"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815\": container with ID starting with fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815 not found: ID does not exist" Sep 10 22:04:37.441056 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.440997 2576 scope.go:117] "RemoveContainer" containerID="7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227" Sep 10 22:04:37.441216 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.441201 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227"} err="failed to get container status \"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227\": rpc error: code = NotFound desc = could not find container \"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227\": container with ID starting with 7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227 not found: ID does not exist" Sep 10 22:04:37.441277 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.441216 2576 scope.go:117] "RemoveContainer" containerID="8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245" Sep 10 22:04:37.441411 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.441396 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245"} err="failed to get container status \"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245\": rpc error: code = NotFound desc = could not find container \"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245\": container with ID starting with 8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245 not found: ID does not exist" Sep 10 22:04:37.441411 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.441411 2576 scope.go:117] "RemoveContainer" containerID="b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c" Sep 10 22:04:37.441652 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.441634 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c"} err="failed to get container status \"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c\": rpc error: code = NotFound desc = could not find container \"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c\": container with ID starting with b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c not found: ID does not exist" Sep 10 22:04:37.441719 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.441653 2576 scope.go:117] "RemoveContainer" containerID="2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff" Sep 10 22:04:37.441992 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.441975 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff"} err="failed to get container status \"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff\": rpc error: code = NotFound desc = could not find container \"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff\": container with ID starting with 2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff not found: ID does not exist" Sep 10 22:04:37.441992 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.441991 2576 scope.go:117] "RemoveContainer" containerID="12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1" Sep 10 22:04:37.442215 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.442191 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1"} err="failed to get container status \"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1\": rpc error: code = NotFound desc = could not find container \"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1\": container with ID starting with 12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1 not found: ID does not exist" Sep 10 22:04:37.442259 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.442216 2576 scope.go:117] "RemoveContainer" containerID="858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990" Sep 10 22:04:37.442458 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.442443 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990"} err="failed to get container status \"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990\": rpc error: code = NotFound desc = could not find container \"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990\": container with ID starting with 858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990 not found: ID does not exist" Sep 10 22:04:37.442497 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.442462 2576 scope.go:117] "RemoveContainer" containerID="fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815" Sep 10 22:04:37.442497 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.442449 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.442957 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.442695 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815"} err="failed to get container status \"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815\": rpc error: code = NotFound desc = could not find container \"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815\": container with ID starting with fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815 not found: ID does not exist" Sep 10 22:04:37.442957 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.442737 2576 scope.go:117] "RemoveContainer" containerID="7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227" Sep 10 22:04:37.443107 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.443084 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227"} err="failed to get container status \"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227\": rpc error: code = NotFound desc = could not find container \"7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227\": container with ID starting with 7e3260034bed350e8cd5dc85c2b50f6e0bffb627d51152f7ac7f28b52a57d227 not found: ID does not exist" Sep 10 22:04:37.443164 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.443110 2576 scope.go:117] "RemoveContainer" containerID="8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245" Sep 10 22:04:37.443380 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.443364 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245"} err="failed to get container status \"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245\": rpc error: code = NotFound desc = could not find container \"8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245\": container with ID starting with 8cfcfb2990ddbf83f39840a928b65e54faec747acce779369b1690901fe15245 not found: ID does not exist" Sep 10 22:04:37.443427 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.443381 2576 scope.go:117] "RemoveContainer" containerID="b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c" Sep 10 22:04:37.443638 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.443618 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c"} err="failed to get container status \"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c\": rpc error: code = NotFound desc = could not find container \"b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c\": container with ID starting with b1cb56e78956572c3513420bfe34b7dbd3c5005a97036dc785ce3a97f8bc008c not found: ID does not exist" Sep 10 22:04:37.443710 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.443640 2576 scope.go:117] "RemoveContainer" containerID="2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff" Sep 10 22:04:37.443936 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.443919 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff"} err="failed to get container status \"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff\": rpc error: code = NotFound desc = could not find container \"2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff\": container with ID starting with 2310490d72e31d5ed845ade6748400a5472b967d3733df4e18792304ff3b51ff not found: ID does not exist" Sep 10 22:04:37.443989 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.443935 2576 scope.go:117] "RemoveContainer" containerID="12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1" Sep 10 22:04:37.444190 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.444167 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1"} err="failed to get container status \"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1\": rpc error: code = NotFound desc = could not find container \"12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1\": container with ID starting with 12ad3a1fd35f4994bc77a7dd9e32f53fe54764db1704ec647fc80629462fdef1 not found: ID does not exist" Sep 10 22:04:37.444190 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.444183 2576 scope.go:117] "RemoveContainer" containerID="858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990" Sep 10 22:04:37.444438 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.444418 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990"} err="failed to get container status \"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990\": rpc error: code = NotFound desc = could not find container \"858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990\": container with ID starting with 858662b29a635ee9a75aaf7947abe1188edc9176b6b45605b207447847481990 not found: ID does not exist" Sep 10 22:04:37.444498 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.444440 2576 scope.go:117] "RemoveContainer" containerID="fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815" Sep 10 22:04:37.444592 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.444577 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 22:04:37.444770 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.444738 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 22:04:37.444770 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.444754 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815"} err="failed to get container status \"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815\": rpc error: code = NotFound desc = could not find container \"fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815\": container with ID starting with fe9e2e5442e9963e53315fcf0ea94529ed9b16a3abec8589d68113fe7c355815 not found: ID does not exist" Sep 10 22:04:37.444938 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.444907 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 22:04:37.445027 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.445010 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 22:04:37.445321 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.445307 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6v9zt\"" Sep 10 22:04:37.445412 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.445395 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 22:04:37.445488 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.445401 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 22:04:37.445567 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.445550 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 22:04:37.445813 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.445784 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 22:04:37.445885 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.445827 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 22:04:37.445934 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.445882 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 22:04:37.446038 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.446022 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dqeor0lierrf3\"" Sep 10 22:04:37.446135 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.446080 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 22:04:37.446195 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.446145 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 22:04:37.446562 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.446540 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 22:04:37.448573 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.448554 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 22:04:37.452096 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.452078 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 22:04:37.458479 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.458456 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 22:04:37.508302 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508202 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.508302 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508243 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.508302 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508265 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jscnr\" (UniqueName: \"kubernetes.io/projected/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-kube-api-access-jscnr\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.508589 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508329 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-web-config\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.508589 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508460 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-config-out\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.508589 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508484 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.508589 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508509 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.508589 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508546 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.508589 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508568 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.508589 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508594 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.508953 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508636 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.508953 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508674 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.508953 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508726 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.508953 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508844 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.508953 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508891 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.508953 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.508924 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.509239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.509015 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-config\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.509239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.509060 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.509239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.509105 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.509239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.509182 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.610486 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.610448 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.610486 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.610487 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.610734 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.610504 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jscnr\" (UniqueName: \"kubernetes.io/projected/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-kube-api-access-jscnr\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.610734 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.610521 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-web-config\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.610734 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.610623 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-config-out\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.610734 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.610656 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.610734 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.610683 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.610734 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.610710 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.611071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.610744 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.611071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.610774 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.611071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.610840 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.611071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.610867 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.611071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.610909 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.611071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.610959 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.611071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.610988 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.611071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.611019 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.611071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.611056 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-config\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.611490 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.611084 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.611490 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.611108 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.611490 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.611151 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.611490 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.611414 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.613078 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.611783 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.613078 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.612211 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.613078 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.612444 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.613078 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.612457 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.613078 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.612845 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.613309 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.613187 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.613873 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.613847 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.614465 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.614438 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.614714 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.614689 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-config-out\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.615139 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.615096 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-web-config\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.615323 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.615299 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.615419 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.615398 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.615999 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.615957 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.616264 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.616244 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.616401 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.616379 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.616437 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.616394 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.616482 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.616461 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-config\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.616633 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.616618 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.619914 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.619893 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jscnr\" (UniqueName: \"kubernetes.io/projected/2c00e9cc-f49a-45f5-a3b2-01bc2146bafa-kube-api-access-jscnr\") pod \"prometheus-k8s-0\" (UID: \"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:37.752979 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:37.752945 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:38.049069 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:38.048723 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 22:04:38.051583 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:04:38.051549 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c00e9cc_f49a_45f5_a3b2_01bc2146bafa.slice/crio-3dfc1359c5a3f937f70923a8384316a6d1c4f45bab2c1595c6620bc436342f52 WatchSource:0}: Error finding container 3dfc1359c5a3f937f70923a8384316a6d1c4f45bab2c1595c6620bc436342f52: Status 404 returned error can't find the container with id 3dfc1359c5a3f937f70923a8384316a6d1c4f45bab2c1595c6620bc436342f52 Sep 10 22:04:38.385767 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:38.385724 2576 generic.go:356] "Generic (PLEG): container finished" podID="2c00e9cc-f49a-45f5-a3b2-01bc2146bafa" containerID="31cbda12d346083eee5db5a214f3d0a6545b2a165458ffc9f0236b5cd7152e2f" exitCode=0 Sep 10 22:04:38.386253 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:38.385830 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa","Type":"ContainerDied","Data":"31cbda12d346083eee5db5a214f3d0a6545b2a165458ffc9f0236b5cd7152e2f"} Sep 10 22:04:38.386253 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:38.385927 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa","Type":"ContainerStarted","Data":"3dfc1359c5a3f937f70923a8384316a6d1c4f45bab2c1595c6620bc436342f52"} Sep 10 22:04:38.438135 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:38.435288 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f13be60c-9156-4e4a-a067-7764705e6564" path="/var/lib/kubelet/pods/f13be60c-9156-4e4a-a067-7764705e6564/volumes" Sep 10 22:04:39.393597 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:39.393557 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" event={"ID":"ff35ebdf-3271-4d7b-8490-dd9c25d9e714","Type":"ContainerStarted","Data":"e2323bfd794a350670b7db6673b8951992169594b54f7ef0db94bccfd13e4fa7"} Sep 10 22:04:39.394152 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:39.393627 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" event={"ID":"ff35ebdf-3271-4d7b-8490-dd9c25d9e714","Type":"ContainerStarted","Data":"175a1ad12695e4ca3a0a520e6a756b05e23d65c6b8c39ee14317b913deadb5c2"} Sep 10 22:04:39.394152 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:39.393660 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" event={"ID":"ff35ebdf-3271-4d7b-8490-dd9c25d9e714","Type":"ContainerStarted","Data":"f3c9e0dd746697f9e40af125121ccf9c3b25eb9ad0b30ba549e2354b7266d545"} Sep 10 22:04:39.397032 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:39.397005 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa","Type":"ContainerStarted","Data":"d6e69bd6ff057cb4713f012abfd8abe5c9a32a2bda38b5847ea695f9aa899370"} Sep 10 22:04:39.397186 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:39.397076 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa","Type":"ContainerStarted","Data":"d3281b0aac31ff25a5a27c1e80005841a35ccc0c97961f3c5ab5981463f4a5be"} Sep 10 22:04:39.397186 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:39.397117 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa","Type":"ContainerStarted","Data":"10306ebcf4d86857e027dc929e57bee2be8746850bc228a3511f450bf3ada1b9"} Sep 10 22:04:39.397186 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:39.397154 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa","Type":"ContainerStarted","Data":"291c57d8c45d80c02fefe61043d5efc62613ac379d6f7a187d4ecebae85d152e"} Sep 10 22:04:39.397343 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:39.397197 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa","Type":"ContainerStarted","Data":"1deb8b74da04e8cf6110afcb6101c47001eb51881e16ce8eba2ab969b4524df2"} Sep 10 22:04:39.397343 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:39.397236 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c00e9cc-f49a-45f5-a3b2-01bc2146bafa","Type":"ContainerStarted","Data":"eff562be8ede7e9b22546af9c289d05414d081e53322ce98db37b47557ace868"} Sep 10 22:04:39.415452 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:39.415328 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6c44fd675d-t98kh" podStartSLOduration=1.651992428 podStartE2EDuration="3.415306644s" totalImagesPullingTime="1.763314216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:04:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:04:39.41339576 +0000 UTC m=+209.535094542" watchObservedRunningTime="2026-09-10 22:04:39.415306644 +0000 UTC m=+209.537005426" Sep 10 22:04:39.440550 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:39.440491 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.418188636 podStartE2EDuration="2.440473466s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.28483ms" isStatefulPod=false podCreationTimestamp="2026-09-10 22:04:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:04:39.438365011 +0000 UTC m=+209.560063793" watchObservedRunningTime="2026-09-10 22:04:39.440473466 +0000 UTC m=+209.562172275" Sep 10 22:04:39.557777 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:39.557743 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:04:39.620174 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:39.620142 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:04:39.701678 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:39.701593 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:04:40.298426 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:40.298387 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:04:40.358127 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:40.358090 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:04:42.753935 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:42.753893 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:04:50.833030 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:50.832999 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:04:50.934231 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:50.934200 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:04:50.994480 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:04:50.994448 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:05:22.035345 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:05:22.035313 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:05:25.367543 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:05:25.367515 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:05:37.753492 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:05:37.753383 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:05:37.769312 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:05:37.769281 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:05:38.648002 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:05:38.647969 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 22:05:46.899298 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:05:46.899257 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:05:46.994167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:05:46.994134 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:05:58.901053 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:05:58.901014 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:05:59.020444 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:05:59.020412 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:06:01.965713 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:06:01.965672 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:06:02.199817 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:06:02.199770 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:06:03.509313 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:06:03.509273 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:06:05.436209 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:06:05.436172 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:06:05.566433 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:06:05.566387 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:06:06.080564 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:06:06.080534 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:06:10.358913 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:06:10.358882 2576 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 22:06:16.161615 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:06:16.161580 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:06:16.261480 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:06:16.261447 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:06:16.331150 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:06:16.331120 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:06:43.153184 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:06:43.153139 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:06:43.286044 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:06:43.286006 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:05.167273 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:05.167185 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:06.799293 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:06.799257 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:08.972193 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:08.972159 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:09.029928 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:09.029898 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:10.629572 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:10.629534 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:10.718997 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:10.718965 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:11.234355 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:11.234325 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:11.337099 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:11.337064 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:13.445518 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:13.445481 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:13.628132 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:13.628098 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:15.966328 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:15.966298 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:16.067834 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:16.067769 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:18.133588 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:18.133549 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jmf4c"] Sep 10 22:07:18.136930 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:18.136906 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmf4c" Sep 10 22:07:18.139065 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:18.139038 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 10 22:07:18.139184 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:18.139105 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 10 22:07:18.139595 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:18.139579 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-l9w5x\"" Sep 10 22:07:18.148769 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:18.148742 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jmf4c"] Sep 10 22:07:18.182183 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:18.182145 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fpjq\" (UniqueName: \"kubernetes.io/projected/6aab66f1-29f9-4dbe-9bb8-3fab3b1b92c4-kube-api-access-4fpjq\") pod \"cert-manager-cainjector-559cdd8c87-jmf4c\" (UID: \"6aab66f1-29f9-4dbe-9bb8-3fab3b1b92c4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmf4c" Sep 10 22:07:18.182371 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:18.182223 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6aab66f1-29f9-4dbe-9bb8-3fab3b1b92c4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jmf4c\" (UID: \"6aab66f1-29f9-4dbe-9bb8-3fab3b1b92c4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmf4c" Sep 10 22:07:18.282845 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:18.282784 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fpjq\" (UniqueName: \"kubernetes.io/projected/6aab66f1-29f9-4dbe-9bb8-3fab3b1b92c4-kube-api-access-4fpjq\") pod \"cert-manager-cainjector-559cdd8c87-jmf4c\" (UID: \"6aab66f1-29f9-4dbe-9bb8-3fab3b1b92c4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmf4c" Sep 10 22:07:18.283025 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:18.282872 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6aab66f1-29f9-4dbe-9bb8-3fab3b1b92c4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jmf4c\" (UID: \"6aab66f1-29f9-4dbe-9bb8-3fab3b1b92c4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmf4c" Sep 10 22:07:18.290856 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:18.290827 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6aab66f1-29f9-4dbe-9bb8-3fab3b1b92c4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jmf4c\" (UID: \"6aab66f1-29f9-4dbe-9bb8-3fab3b1b92c4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmf4c" Sep 10 22:07:18.290990 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:18.290970 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fpjq\" (UniqueName: \"kubernetes.io/projected/6aab66f1-29f9-4dbe-9bb8-3fab3b1b92c4-kube-api-access-4fpjq\") pod \"cert-manager-cainjector-559cdd8c87-jmf4c\" (UID: \"6aab66f1-29f9-4dbe-9bb8-3fab3b1b92c4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmf4c" Sep 10 22:07:18.460063 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:18.459976 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmf4c" Sep 10 22:07:18.732410 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:18.732327 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jmf4c"] Sep 10 22:07:18.737026 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:07:18.736989 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6aab66f1_29f9_4dbe_9bb8_3fab3b1b92c4.slice/crio-9a8dae484a888d85c1b1d07f0019cde668f8f2fe99ab0d9a3e077239f2c17d83 WatchSource:0}: Error finding container 9a8dae484a888d85c1b1d07f0019cde668f8f2fe99ab0d9a3e077239f2c17d83: Status 404 returned error can't find the container with id 9a8dae484a888d85c1b1d07f0019cde668f8f2fe99ab0d9a3e077239f2c17d83 Sep 10 22:07:18.738998 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:18.738979 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 22:07:19.029840 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:19.029730 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmf4c" event={"ID":"6aab66f1-29f9-4dbe-9bb8-3fab3b1b92c4","Type":"ContainerStarted","Data":"9a8dae484a888d85c1b1d07f0019cde668f8f2fe99ab0d9a3e077239f2c17d83"} Sep 10 22:07:21.640760 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:21.640721 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:22.876885 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:22.876850 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:22.940391 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:22.940358 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:23.052188 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:23.052144 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmf4c" event={"ID":"6aab66f1-29f9-4dbe-9bb8-3fab3b1b92c4","Type":"ContainerStarted","Data":"24a01ef5b0aba8e46e94665e00ca540bb95ad3a5205b772bcccb9cc680c2faa7"} Sep 10 22:07:23.068596 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:23.068542 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmf4c" podStartSLOduration=1.717305173 podStartE2EDuration="5.068525961s" totalImagesPullingTime="3.351220788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:07:23.067663843 +0000 UTC m=+373.189362625" watchObservedRunningTime="2026-09-10 22:07:23.068525961 +0000 UTC m=+373.190224745" Sep 10 22:07:26.373152 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:26.373121 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:26.473893 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:26.473864 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:32.730497 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:32.730464 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:32.831448 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:32.831415 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:34.435050 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:34.435015 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:47.376202 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:47.376165 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:47.473039 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:47.473006 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:48.003875 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.003774 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:48.350341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.350304 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:48.441465 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.441434 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:48.503598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.503563 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg"] Sep 10 22:07:48.508157 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.508136 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.512079 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.512044 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 22:07:48.512079 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.512065 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 22:07:48.512299 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.512113 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 22:07:48.512370 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.512356 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 22:07:48.513678 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.513644 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5r2tg\"" Sep 10 22:07:48.536852 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.536816 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg"] Sep 10 22:07:48.546713 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.546678 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/22914f3d-9e83-45d0-a503-f995e39dc50c-webhook-cert\") pod \"opendatahub-operator-controller-manager-76d44b6d5b-v5mwg\" (UID: \"22914f3d-9e83-45d0-a503-f995e39dc50c\") " pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.546914 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.546746 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/22914f3d-9e83-45d0-a503-f995e39dc50c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76d44b6d5b-v5mwg\" (UID: \"22914f3d-9e83-45d0-a503-f995e39dc50c\") " pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.546914 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.546822 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22914f3d-9e83-45d0-a503-f995e39dc50c-tmp\") pod \"opendatahub-operator-controller-manager-76d44b6d5b-v5mwg\" (UID: \"22914f3d-9e83-45d0-a503-f995e39dc50c\") " pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.546914 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.546864 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/22914f3d-9e83-45d0-a503-f995e39dc50c-manifests\") pod \"opendatahub-operator-controller-manager-76d44b6d5b-v5mwg\" (UID: \"22914f3d-9e83-45d0-a503-f995e39dc50c\") " pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.547082 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.546955 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdfl5\" (UniqueName: \"kubernetes.io/projected/22914f3d-9e83-45d0-a503-f995e39dc50c-kube-api-access-qdfl5\") pod \"opendatahub-operator-controller-manager-76d44b6d5b-v5mwg\" (UID: \"22914f3d-9e83-45d0-a503-f995e39dc50c\") " pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.647672 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.647570 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/22914f3d-9e83-45d0-a503-f995e39dc50c-webhook-cert\") pod \"opendatahub-operator-controller-manager-76d44b6d5b-v5mwg\" (UID: \"22914f3d-9e83-45d0-a503-f995e39dc50c\") " pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.647672 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.647625 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/22914f3d-9e83-45d0-a503-f995e39dc50c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76d44b6d5b-v5mwg\" (UID: \"22914f3d-9e83-45d0-a503-f995e39dc50c\") " pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.647932 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.647673 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22914f3d-9e83-45d0-a503-f995e39dc50c-tmp\") pod \"opendatahub-operator-controller-manager-76d44b6d5b-v5mwg\" (UID: \"22914f3d-9e83-45d0-a503-f995e39dc50c\") " pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.647932 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.647707 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/22914f3d-9e83-45d0-a503-f995e39dc50c-manifests\") pod \"opendatahub-operator-controller-manager-76d44b6d5b-v5mwg\" (UID: \"22914f3d-9e83-45d0-a503-f995e39dc50c\") " pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.647932 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.647755 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdfl5\" (UniqueName: \"kubernetes.io/projected/22914f3d-9e83-45d0-a503-f995e39dc50c-kube-api-access-qdfl5\") pod \"opendatahub-operator-controller-manager-76d44b6d5b-v5mwg\" (UID: \"22914f3d-9e83-45d0-a503-f995e39dc50c\") " pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.648191 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.648167 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/22914f3d-9e83-45d0-a503-f995e39dc50c-manifests\") pod \"opendatahub-operator-controller-manager-76d44b6d5b-v5mwg\" (UID: \"22914f3d-9e83-45d0-a503-f995e39dc50c\") " pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.650324 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.650295 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/22914f3d-9e83-45d0-a503-f995e39dc50c-webhook-cert\") pod \"opendatahub-operator-controller-manager-76d44b6d5b-v5mwg\" (UID: \"22914f3d-9e83-45d0-a503-f995e39dc50c\") " pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.650461 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.650338 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22914f3d-9e83-45d0-a503-f995e39dc50c-tmp\") pod \"opendatahub-operator-controller-manager-76d44b6d5b-v5mwg\" (UID: \"22914f3d-9e83-45d0-a503-f995e39dc50c\") " pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.650637 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.650620 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/22914f3d-9e83-45d0-a503-f995e39dc50c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76d44b6d5b-v5mwg\" (UID: \"22914f3d-9e83-45d0-a503-f995e39dc50c\") " pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.651080 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.651058 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:48.661104 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.661081 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdfl5\" (UniqueName: \"kubernetes.io/projected/22914f3d-9e83-45d0-a503-f995e39dc50c-kube-api-access-qdfl5\") pod \"opendatahub-operator-controller-manager-76d44b6d5b-v5mwg\" (UID: \"22914f3d-9e83-45d0-a503-f995e39dc50c\") " pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.747761 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.747726 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:48.819555 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.819515 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:48.850344 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.850311 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:48.967025 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:48.966937 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:49.111252 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:49.111217 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg"] Sep 10 22:07:49.115700 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:07:49.115671 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22914f3d_9e83_45d0_a503_f995e39dc50c.slice/crio-ade1640a5ddb16924cdf070beb1f2f3367bac16f66e9156324f94596e3eb8d4b WatchSource:0}: Error finding container ade1640a5ddb16924cdf070beb1f2f3367bac16f66e9156324f94596e3eb8d4b: Status 404 returned error can't find the container with id ade1640a5ddb16924cdf070beb1f2f3367bac16f66e9156324f94596e3eb8d4b Sep 10 22:07:49.125219 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:49.125188 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:49.156531 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:49.156496 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" event={"ID":"22914f3d-9e83-45d0-a503-f995e39dc50c","Type":"ContainerStarted","Data":"ade1640a5ddb16924cdf070beb1f2f3367bac16f66e9156324f94596e3eb8d4b"} Sep 10 22:07:49.207581 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:49.207546 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:52.173151 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:52.173113 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" event={"ID":"22914f3d-9e83-45d0-a503-f995e39dc50c","Type":"ContainerStarted","Data":"7b199902c1422f5f8267379285c365ea455b81ee8829cb6a02081e96b261bfbd"} Sep 10 22:07:54.182649 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:54.182615 2576 generic.go:356] "Generic (PLEG): container finished" podID="22914f3d-9e83-45d0-a503-f995e39dc50c" containerID="7b199902c1422f5f8267379285c365ea455b81ee8829cb6a02081e96b261bfbd" exitCode=0 Sep 10 22:07:54.183083 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:54.182685 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" event={"ID":"22914f3d-9e83-45d0-a503-f995e39dc50c","Type":"ContainerDied","Data":"7b199902c1422f5f8267379285c365ea455b81ee8829cb6a02081e96b261bfbd"} Sep 10 22:07:54.708714 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:54.708667 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:54.827449 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:54.827414 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:55.189012 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:55.188979 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" event={"ID":"22914f3d-9e83-45d0-a503-f995e39dc50c","Type":"ContainerStarted","Data":"e1e15abbc65a2f38f5f54c8bd8fc3d93911d6c35f7bd35de278c056538b79612"} Sep 10 22:07:55.189414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:55.189294 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:07:55.212395 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:55.212341 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" podStartSLOduration=2.792665307 podStartE2EDuration="7.212325278s" totalImagesPullingTime="3.34091197s" totalInitContainerRuntime="1.078748001s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:07:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:07:55.212135558 +0000 UTC m=+405.333834339" watchObservedRunningTime="2026-09-10 22:07:55.212325278 +0000 UTC m=+405.334024060" Sep 10 22:07:58.846597 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:58.846563 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:07:58.945919 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:07:58.945872 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:06.195645 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:06.195608 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-76d44b6d5b-v5mwg" Sep 10 22:08:07.801622 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:07.801590 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:07.977762 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:07.977729 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:08.388677 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:08.388645 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:08.769567 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:08.769484 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:10.152335 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:10.152305 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:10.239960 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:10.239929 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:11.347256 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:11.347215 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:14.844443 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:14.844409 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:14.917277 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:14.917243 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:15.672601 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:15.672566 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:15.916210 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:15.916183 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:16.086080 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:16.086043 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:19.136494 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:19.136456 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:19.234628 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:19.234593 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:19.335111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:19.335076 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:19.741685 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:19.741651 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:19.889376 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:19.889343 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:20.035447 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:20.035340 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:25.209850 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.209782 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7"] Sep 10 22:08:25.246524 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.246486 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7"] Sep 10 22:08:25.246708 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.246622 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:25.249507 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.249481 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 10 22:08:25.249665 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.249568 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4whms\"" Sep 10 22:08:25.249665 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.249491 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 10 22:08:25.249665 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.249529 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 10 22:08:25.249665 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.249534 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 10 22:08:25.249940 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.249549 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 10 22:08:25.282074 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.282038 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5687ceb7-b470-42d1-85ac-1b5993671b35-metrics-cert\") pod \"lws-controller-manager-56f58b8775-f8cn7\" (UID: \"5687ceb7-b470-42d1-85ac-1b5993671b35\") " pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:25.282257 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.282105 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prs7g\" (UniqueName: \"kubernetes.io/projected/5687ceb7-b470-42d1-85ac-1b5993671b35-kube-api-access-prs7g\") pod \"lws-controller-manager-56f58b8775-f8cn7\" (UID: \"5687ceb7-b470-42d1-85ac-1b5993671b35\") " pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:25.282257 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.282151 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5687ceb7-b470-42d1-85ac-1b5993671b35-manager-config\") pod \"lws-controller-manager-56f58b8775-f8cn7\" (UID: \"5687ceb7-b470-42d1-85ac-1b5993671b35\") " pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:25.282257 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.282179 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5687ceb7-b470-42d1-85ac-1b5993671b35-cert\") pod \"lws-controller-manager-56f58b8775-f8cn7\" (UID: \"5687ceb7-b470-42d1-85ac-1b5993671b35\") " pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:25.383036 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.382993 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prs7g\" (UniqueName: \"kubernetes.io/projected/5687ceb7-b470-42d1-85ac-1b5993671b35-kube-api-access-prs7g\") pod \"lws-controller-manager-56f58b8775-f8cn7\" (UID: \"5687ceb7-b470-42d1-85ac-1b5993671b35\") " pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:25.383036 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.383045 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5687ceb7-b470-42d1-85ac-1b5993671b35-manager-config\") pod \"lws-controller-manager-56f58b8775-f8cn7\" (UID: \"5687ceb7-b470-42d1-85ac-1b5993671b35\") " pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:25.383293 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.383060 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5687ceb7-b470-42d1-85ac-1b5993671b35-cert\") pod \"lws-controller-manager-56f58b8775-f8cn7\" (UID: \"5687ceb7-b470-42d1-85ac-1b5993671b35\") " pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:25.383293 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.383102 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5687ceb7-b470-42d1-85ac-1b5993671b35-metrics-cert\") pod \"lws-controller-manager-56f58b8775-f8cn7\" (UID: \"5687ceb7-b470-42d1-85ac-1b5993671b35\") " pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:25.383789 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.383758 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5687ceb7-b470-42d1-85ac-1b5993671b35-manager-config\") pod \"lws-controller-manager-56f58b8775-f8cn7\" (UID: \"5687ceb7-b470-42d1-85ac-1b5993671b35\") " pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:25.385615 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.385590 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5687ceb7-b470-42d1-85ac-1b5993671b35-cert\") pod \"lws-controller-manager-56f58b8775-f8cn7\" (UID: \"5687ceb7-b470-42d1-85ac-1b5993671b35\") " pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:25.385727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.385638 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5687ceb7-b470-42d1-85ac-1b5993671b35-metrics-cert\") pod \"lws-controller-manager-56f58b8775-f8cn7\" (UID: \"5687ceb7-b470-42d1-85ac-1b5993671b35\") " pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:25.393136 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.393110 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prs7g\" (UniqueName: \"kubernetes.io/projected/5687ceb7-b470-42d1-85ac-1b5993671b35-kube-api-access-prs7g\") pod \"lws-controller-manager-56f58b8775-f8cn7\" (UID: \"5687ceb7-b470-42d1-85ac-1b5993671b35\") " pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:25.556730 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.556634 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:25.829005 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.828943 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7"] Sep 10 22:08:25.831642 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:08:25.831613 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5687ceb7_b470_42d1_85ac_1b5993671b35.slice/crio-7098c5f1305e6609aaab9a09fd6773192fbcedbd49252ea19e55276df823fa7e WatchSource:0}: Error finding container 7098c5f1305e6609aaab9a09fd6773192fbcedbd49252ea19e55276df823fa7e: Status 404 returned error can't find the container with id 7098c5f1305e6609aaab9a09fd6773192fbcedbd49252ea19e55276df823fa7e Sep 10 22:08:25.904058 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.904022 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5"] Sep 10 22:08:25.906844 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.906822 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:25.909340 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.909311 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 10 22:08:25.909485 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.909349 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 10 22:08:25.909629 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.909611 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 10 22:08:25.909686 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.909665 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 22:08:25.909818 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.909788 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-p8mrv\"" Sep 10 22:08:25.910043 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.910016 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 10 22:08:25.910129 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.910068 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 22:08:25.920634 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.920606 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5"] Sep 10 22:08:25.986391 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.986357 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/033c5b0a-8df9-4838-9617-5be1317d86ca-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:25.986590 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.986403 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/033c5b0a-8df9-4838-9617-5be1317d86ca-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:25.986590 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.986457 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/033c5b0a-8df9-4838-9617-5be1317d86ca-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:25.986590 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.986505 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/033c5b0a-8df9-4838-9617-5be1317d86ca-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:25.986590 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.986576 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/033c5b0a-8df9-4838-9617-5be1317d86ca-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:25.986786 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.986619 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/033c5b0a-8df9-4838-9617-5be1317d86ca-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:25.986786 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:25.986660 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcbsf\" (UniqueName: \"kubernetes.io/projected/033c5b0a-8df9-4838-9617-5be1317d86ca-kube-api-access-qcbsf\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.087183 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.087090 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/033c5b0a-8df9-4838-9617-5be1317d86ca-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.087183 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.087127 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcbsf\" (UniqueName: \"kubernetes.io/projected/033c5b0a-8df9-4838-9617-5be1317d86ca-kube-api-access-qcbsf\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.087426 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.087253 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/033c5b0a-8df9-4838-9617-5be1317d86ca-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.087426 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.087307 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/033c5b0a-8df9-4838-9617-5be1317d86ca-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.087426 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.087331 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/033c5b0a-8df9-4838-9617-5be1317d86ca-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.087426 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.087359 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/033c5b0a-8df9-4838-9617-5be1317d86ca-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.087426 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.087413 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/033c5b0a-8df9-4838-9617-5be1317d86ca-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.087961 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.087941 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/033c5b0a-8df9-4838-9617-5be1317d86ca-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.089568 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.089544 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/033c5b0a-8df9-4838-9617-5be1317d86ca-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.089661 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.089630 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/033c5b0a-8df9-4838-9617-5be1317d86ca-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.089941 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.089925 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/033c5b0a-8df9-4838-9617-5be1317d86ca-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.089979 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.089961 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/033c5b0a-8df9-4838-9617-5be1317d86ca-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.102484 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.102451 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/033c5b0a-8df9-4838-9617-5be1317d86ca-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.102973 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.102954 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcbsf\" (UniqueName: \"kubernetes.io/projected/033c5b0a-8df9-4838-9617-5be1317d86ca-kube-api-access-qcbsf\") pod \"istiod-openshift-gateway-5677ccb49b-tw2s5\" (UID: \"033c5b0a-8df9-4838-9617-5be1317d86ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.216926 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.216888 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:26.328612 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.328578 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" event={"ID":"5687ceb7-b470-42d1-85ac-1b5993671b35","Type":"ContainerStarted","Data":"7098c5f1305e6609aaab9a09fd6773192fbcedbd49252ea19e55276df823fa7e"} Sep 10 22:08:26.505404 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:26.505362 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5"] Sep 10 22:08:26.508241 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:08:26.508209 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod033c5b0a_8df9_4838_9617_5be1317d86ca.slice/crio-21940e53dbc692b998697ac0b258b43cd317a5b885d35c21548dc00705195142 WatchSource:0}: Error finding container 21940e53dbc692b998697ac0b258b43cd317a5b885d35c21548dc00705195142: Status 404 returned error can't find the container with id 21940e53dbc692b998697ac0b258b43cd317a5b885d35c21548dc00705195142 Sep 10 22:08:27.338047 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:27.338005 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" event={"ID":"033c5b0a-8df9-4838-9617-5be1317d86ca","Type":"ContainerStarted","Data":"21940e53dbc692b998697ac0b258b43cd317a5b885d35c21548dc00705195142"} Sep 10 22:08:29.082125 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:29.082088 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:29.203089 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:29.203053 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:29.332284 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:29.332202 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:29.417597 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:29.417563 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:29.545762 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:29.545727 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 22:08:29.545868 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:29.545795 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 22:08:30.354982 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:30.354940 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" event={"ID":"033c5b0a-8df9-4838-9617-5be1317d86ca","Type":"ContainerStarted","Data":"f8fbd97dd92b7a4978ca337be7ad034aeff72d4c11371903468dab60f47e9ce8"} Sep 10 22:08:30.355454 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:30.355113 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:30.356609 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:30.356582 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" event={"ID":"5687ceb7-b470-42d1-85ac-1b5993671b35","Type":"ContainerStarted","Data":"2c28cd6ce8f72038a2218c917a0010ad6ae4826680252bbdaf63b25cdc53ee31"} Sep 10 22:08:30.356850 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:30.356832 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:30.357126 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:30.357108 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" Sep 10 22:08:30.379345 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:30.379316 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:30.383403 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:30.383353 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tw2s5" podStartSLOduration=2.348517491 podStartE2EDuration="5.383335147s" totalImagesPullingTime="3.034817656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:08:30.38106071 +0000 UTC m=+440.502759491" watchObservedRunningTime="2026-09-10 22:08:30.383335147 +0000 UTC m=+440.505033931" Sep 10 22:08:30.402576 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:30.402519 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" podStartSLOduration=1.691126715 podStartE2EDuration="5.40250155s" totalImagesPullingTime="3.711374835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:08:30.400599974 +0000 UTC m=+440.522298755" watchObservedRunningTime="2026-09-10 22:08:30.40250155 +0000 UTC m=+440.524200332" Sep 10 22:08:30.485403 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:30.485308 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:30.613962 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:30.613876 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:41.363240 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:41.363161 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-56f58b8775-f8cn7" Sep 10 22:08:46.049217 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:46.049182 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:57.043149 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:57.043114 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:08:57.136271 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:08:57.136235 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:20.078016 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:20.077980 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:20.206011 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:20.205976 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:35.513838 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:35.513795 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:36.307354 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:36.307323 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:37.113507 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:37.113477 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:39.448541 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:39.448508 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:39.550569 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:39.550536 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:40.331155 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:40.331117 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:40.774030 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:40.773947 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:40.985842 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:40.985792 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:43.334618 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:43.334578 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:45.179144 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:45.179114 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:45.322440 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:45.322400 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:46.650492 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:46.650460 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:48.034252 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:48.034171 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:48.164034 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:48.164003 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:50.526514 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:50.526481 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:50.626312 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:50.626279 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:50.912318 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:50.912284 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:50.971530 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:50.971502 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:51.520210 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:51.520179 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:51.587949 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:51.587918 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:09:59.119889 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:59.119854 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj"] Sep 10 22:09:59.122325 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:59.122303 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" Sep 10 22:09:59.123240 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:59.123206 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q62vj\" (UID: \"77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" Sep 10 22:09:59.123359 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:59.123278 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qkxx\" (UniqueName: \"kubernetes.io/projected/77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d-kube-api-access-9qkxx\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q62vj\" (UID: \"77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" Sep 10 22:09:59.124718 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:59.124694 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 10 22:09:59.124794 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:59.124694 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 10 22:09:59.128530 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:59.128509 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6bd9z\"" Sep 10 22:09:59.140001 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:59.139972 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj"] Sep 10 22:09:59.223834 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:59.223766 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qkxx\" (UniqueName: \"kubernetes.io/projected/77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d-kube-api-access-9qkxx\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q62vj\" (UID: \"77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" Sep 10 22:09:59.224028 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:59.223940 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q62vj\" (UID: \"77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" Sep 10 22:09:59.224283 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:59.224265 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q62vj\" (UID: \"77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" Sep 10 22:09:59.248378 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:59.248344 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qkxx\" (UniqueName: \"kubernetes.io/projected/77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d-kube-api-access-9qkxx\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q62vj\" (UID: \"77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" Sep 10 22:09:59.432944 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:59.432825 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" Sep 10 22:09:59.712313 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:59.712289 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj"] Sep 10 22:09:59.714691 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:09:59.714663 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b5b1d8_c52c_4e6a_b99c_46ce7b53aa9d.slice/crio-c2542d3867f8a071e4d6ca1d92d94f00435029307adfe1f66afd17586ce5e4b7 WatchSource:0}: Error finding container c2542d3867f8a071e4d6ca1d92d94f00435029307adfe1f66afd17586ce5e4b7: Status 404 returned error can't find the container with id c2542d3867f8a071e4d6ca1d92d94f00435029307adfe1f66afd17586ce5e4b7 Sep 10 22:09:59.753080 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:09:59.753045 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" event={"ID":"77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d","Type":"ContainerStarted","Data":"c2542d3867f8a071e4d6ca1d92d94f00435029307adfe1f66afd17586ce5e4b7"} Sep 10 22:10:01.753534 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:01.753494 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:01.862672 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:01.862641 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:02.321066 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:02.320974 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:02.389841 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:02.389779 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:03.780667 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:03.780561 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" event={"ID":"77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d","Type":"ContainerStarted","Data":"db935b4a69e950d226875af708d0e48b364089a9a1211c8f57391aa274dde4f2"} Sep 10 22:10:03.781096 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:03.780678 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" Sep 10 22:10:03.801307 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:03.801252 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" podStartSLOduration=1.022422336 podStartE2EDuration="4.801234104s" totalImagesPullingTime="3.778811768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:09:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:10:03.7985838 +0000 UTC m=+533.920282654" watchObservedRunningTime="2026-09-10 22:10:03.801234104 +0000 UTC m=+533.922932885" Sep 10 22:10:06.121982 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:06.121945 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:06.220392 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:06.220364 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:10.122186 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:10.122142 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:10.223116 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:10.223069 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:14.787973 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:14.787938 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" Sep 10 22:10:15.538299 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:15.538267 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:15.637146 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:15.637117 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:15.737908 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:15.737880 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:15.839459 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:15.839432 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:16.720982 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:16.720924 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:16.780166 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:16.780127 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:24.331281 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:24.331231 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:24.440404 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:24.440345 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:25.564310 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:25.564277 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:26.786009 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:26.785964 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:26.885979 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:26.885949 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:31.170076 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:31.170027 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:31.276096 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:31.276064 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:31.527210 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:31.527125 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:31.637511 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:31.637474 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:38.444954 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:38.444919 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:38.562098 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:38.562062 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:38.627273 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:38.627242 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:39.277876 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:39.277837 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:39.379369 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:39.379335 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:45.767963 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:45.767927 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:54.529232 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:54.529189 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:54.629450 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:54.629418 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:10:55.003672 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:10:55.003641 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:03.128271 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:03.128236 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:04.132470 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:04.132436 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:04.194074 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:04.194045 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:04.886691 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:04.886660 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:05.001960 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:05.001927 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:11.581090 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.581056 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:11.666037 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.666004 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7769777fc8-kk5x8"] Sep 10 22:11:11.668354 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.668334 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:11.672084 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.672059 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 10 22:11:11.672435 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.672417 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 10 22:11:11.672679 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.672658 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 10 22:11:11.673732 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.673715 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5h96d\"" Sep 10 22:11:11.698943 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.698911 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7769777fc8-kk5x8"] Sep 10 22:11:11.718864 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.718835 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:11.773491 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.773433 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-webhook-cert\") pod \"maas-controller-7769777fc8-kk5x8\" (UID: \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\") " pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:11.773491 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.773498 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-metrics-certs\") pod \"maas-controller-7769777fc8-kk5x8\" (UID: \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\") " pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:11.773726 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.773619 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qrdq\" (UniqueName: \"kubernetes.io/projected/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-kube-api-access-9qrdq\") pod \"maas-controller-7769777fc8-kk5x8\" (UID: \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\") " pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:11.859404 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.859314 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-85457bf75f-p8grg"] Sep 10 22:11:11.862608 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.862579 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:11:11.871793 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.871770 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:11.874047 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.874020 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qrdq\" (UniqueName: \"kubernetes.io/projected/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-kube-api-access-9qrdq\") pod \"maas-controller-7769777fc8-kk5x8\" (UID: \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\") " pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:11.874192 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.874102 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-webhook-cert\") pod \"maas-controller-7769777fc8-kk5x8\" (UID: \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\") " pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:11.874192 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.874141 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-metrics-certs\") pod \"maas-controller-7769777fc8-kk5x8\" (UID: \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\") " pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:11.874314 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:11:11.874190 2576 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 10 22:11:11.874314 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:11:11.874252 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-webhook-cert podName:fbb1c431-8faa-4b8e-8ded-2ea86f42fbec nodeName:}" failed. No retries permitted until 2026-09-10 22:11:12.374231586 +0000 UTC m=+602.495930345 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-webhook-cert") pod "maas-controller-7769777fc8-kk5x8" (UID: "fbb1c431-8faa-4b8e-8ded-2ea86f42fbec") : secret "maas-controller-webhook-cert" not found Sep 10 22:11:11.876620 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.876598 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-metrics-certs\") pod \"maas-controller-7769777fc8-kk5x8\" (UID: \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\") " pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:11.880226 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.880200 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85457bf75f-p8grg"] Sep 10 22:11:11.887436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.887411 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qrdq\" (UniqueName: \"kubernetes.io/projected/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-kube-api-access-9qrdq\") pod \"maas-controller-7769777fc8-kk5x8\" (UID: \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\") " pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:11.974753 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.974712 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdzlw\" (UniqueName: \"kubernetes.io/projected/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-kube-api-access-bdzlw\") pod \"maas-controller-85457bf75f-p8grg\" (UID: \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\") " pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:11:11.974753 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.974757 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-metrics-certs\") pod \"maas-controller-85457bf75f-p8grg\" (UID: \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\") " pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:11:11.975097 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:11.974928 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-webhook-cert\") pod \"maas-controller-85457bf75f-p8grg\" (UID: \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\") " pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:11:12.076361 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:12.076319 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-webhook-cert\") pod \"maas-controller-85457bf75f-p8grg\" (UID: \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\") " pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:11:12.076533 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:12.076387 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdzlw\" (UniqueName: \"kubernetes.io/projected/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-kube-api-access-bdzlw\") pod \"maas-controller-85457bf75f-p8grg\" (UID: \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\") " pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:11:12.076533 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:12.076410 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-metrics-certs\") pod \"maas-controller-85457bf75f-p8grg\" (UID: \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\") " pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:11:12.076533 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:11:12.076502 2576 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 10 22:11:12.076703 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:11:12.076592 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-webhook-cert podName:58b36abc-694b-41f4-8f3a-a7d1263fa6f2 nodeName:}" failed. No retries permitted until 2026-09-10 22:11:12.576570846 +0000 UTC m=+602.698269626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-webhook-cert") pod "maas-controller-85457bf75f-p8grg" (UID: "58b36abc-694b-41f4-8f3a-a7d1263fa6f2") : secret "maas-controller-webhook-cert" not found Sep 10 22:11:12.078894 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:12.078864 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-metrics-certs\") pod \"maas-controller-85457bf75f-p8grg\" (UID: \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\") " pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:11:12.087039 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:12.087014 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdzlw\" (UniqueName: \"kubernetes.io/projected/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-kube-api-access-bdzlw\") pod \"maas-controller-85457bf75f-p8grg\" (UID: \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\") " pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:11:12.379066 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:12.379023 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-webhook-cert\") pod \"maas-controller-7769777fc8-kk5x8\" (UID: \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\") " pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:12.381486 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:12.381460 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-webhook-cert\") pod \"maas-controller-7769777fc8-kk5x8\" (UID: \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\") " pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:12.389936 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:12.389911 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:12.578591 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:12.578549 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:12.580450 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:12.580425 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-webhook-cert\") pod \"maas-controller-85457bf75f-p8grg\" (UID: \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\") " pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:11:12.582909 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:12.582889 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-webhook-cert\") pod \"maas-controller-85457bf75f-p8grg\" (UID: \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\") " pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:11:12.775465 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:12.775374 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:11:12.853262 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:12.853218 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7769777fc8-kk5x8"] Sep 10 22:11:12.856213 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:11:12.856182 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbb1c431_8faa_4b8e_8ded_2ea86f42fbec.slice/crio-a000d8964bbe1f0ced212e818d63314b4e10b331b6c30c7db6e27decb7517580 WatchSource:0}: Error finding container a000d8964bbe1f0ced212e818d63314b4e10b331b6c30c7db6e27decb7517580: Status 404 returned error can't find the container with id a000d8964bbe1f0ced212e818d63314b4e10b331b6c30c7db6e27decb7517580 Sep 10 22:11:13.047999 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:13.047970 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85457bf75f-p8grg"] Sep 10 22:11:13.050734 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:11:13.050700 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58b36abc_694b_41f4_8f3a_a7d1263fa6f2.slice/crio-8882513a9c7b6b7ccb56895bf1b0cdcff5020afe20436938b4d6bb982ae52168 WatchSource:0}: Error finding container 8882513a9c7b6b7ccb56895bf1b0cdcff5020afe20436938b4d6bb982ae52168: Status 404 returned error can't find the container with id 8882513a9c7b6b7ccb56895bf1b0cdcff5020afe20436938b4d6bb982ae52168 Sep 10 22:11:13.104990 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:13.104950 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85457bf75f-p8grg" event={"ID":"58b36abc-694b-41f4-8f3a-a7d1263fa6f2","Type":"ContainerStarted","Data":"8882513a9c7b6b7ccb56895bf1b0cdcff5020afe20436938b4d6bb982ae52168"} Sep 10 22:11:13.106176 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:13.106149 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7769777fc8-kk5x8" event={"ID":"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec","Type":"ContainerStarted","Data":"a000d8964bbe1f0ced212e818d63314b4e10b331b6c30c7db6e27decb7517580"} Sep 10 22:11:15.866042 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:15.866005 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:16.021602 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:16.021556 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:16.125716 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:16.125615 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85457bf75f-p8grg" event={"ID":"58b36abc-694b-41f4-8f3a-a7d1263fa6f2","Type":"ContainerStarted","Data":"91e71e51f7a9e3fa688e0a7ef4032bf54b0ba0786d0086453eea206f81241a75"} Sep 10 22:11:16.125899 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:16.125732 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:11:16.127418 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:16.127380 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7769777fc8-kk5x8" event={"ID":"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec","Type":"ContainerStarted","Data":"a6e6edf40806d580f1c5da1af33b026fff1c7aff1205cfef6709df4d6e014772"} Sep 10 22:11:16.127553 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:16.127475 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:16.146985 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:16.146932 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-85457bf75f-p8grg" podStartSLOduration=2.650139845 podStartE2EDuration="5.146913593s" totalImagesPullingTime="2.496773748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:11:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:11:16.14507277 +0000 UTC m=+606.266771549" watchObservedRunningTime="2026-09-10 22:11:16.146913593 +0000 UTC m=+606.268612375" Sep 10 22:11:16.169432 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:16.169263 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7769777fc8-kk5x8" podStartSLOduration=2.467862378 podStartE2EDuration="5.169248263s" totalImagesPullingTime="2.701385885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:11:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:11:16.166656059 +0000 UTC m=+606.288354840" watchObservedRunningTime="2026-09-10 22:11:16.169248263 +0000 UTC m=+606.290947045" Sep 10 22:11:27.134960 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:27.134926 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:11:27.135522 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:27.135023 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:27.196259 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:27.196224 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7769777fc8-kk5x8"] Sep 10 22:11:27.196502 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:27.196453 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7769777fc8-kk5x8" podUID="fbb1c431-8faa-4b8e-8ded-2ea86f42fbec" containerName="manager" containerID="cri-o://a6e6edf40806d580f1c5da1af33b026fff1c7aff1205cfef6709df4d6e014772" gracePeriod=30 Sep 10 22:11:27.441842 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:27.441812 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:27.520733 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:27.520696 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-metrics-certs\") pod \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\" (UID: \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\") " Sep 10 22:11:27.520733 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:27.520740 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qrdq\" (UniqueName: \"kubernetes.io/projected/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-kube-api-access-9qrdq\") pod \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\" (UID: \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\") " Sep 10 22:11:27.520970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:27.520772 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-webhook-cert\") pod \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\" (UID: \"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec\") " Sep 10 22:11:27.523117 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:27.523087 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fbb1c431-8faa-4b8e-8ded-2ea86f42fbec" (UID: "fbb1c431-8faa-4b8e-8ded-2ea86f42fbec"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:11:27.523254 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:27.523081 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fbb1c431-8faa-4b8e-8ded-2ea86f42fbec" (UID: "fbb1c431-8faa-4b8e-8ded-2ea86f42fbec"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:11:27.523254 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:27.523161 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-kube-api-access-9qrdq" (OuterVolumeSpecName: "kube-api-access-9qrdq") pod "fbb1c431-8faa-4b8e-8ded-2ea86f42fbec" (UID: "fbb1c431-8faa-4b8e-8ded-2ea86f42fbec"). InnerVolumeSpecName "kube-api-access-9qrdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:11:27.621958 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:27.621922 2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-metrics-certs\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:11:27.621958 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:27.621951 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qrdq\" (UniqueName: \"kubernetes.io/projected/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-kube-api-access-9qrdq\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:11:27.621958 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:27.621962 2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec-webhook-cert\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:11:28.185892 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:28.185853 2576 generic.go:356] "Generic (PLEG): container finished" podID="fbb1c431-8faa-4b8e-8ded-2ea86f42fbec" containerID="a6e6edf40806d580f1c5da1af33b026fff1c7aff1205cfef6709df4d6e014772" exitCode=0 Sep 10 22:11:28.186326 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:28.185901 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7769777fc8-kk5x8" Sep 10 22:11:28.186326 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:28.185922 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7769777fc8-kk5x8" event={"ID":"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec","Type":"ContainerDied","Data":"a6e6edf40806d580f1c5da1af33b026fff1c7aff1205cfef6709df4d6e014772"} Sep 10 22:11:28.186326 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:28.185988 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7769777fc8-kk5x8" event={"ID":"fbb1c431-8faa-4b8e-8ded-2ea86f42fbec","Type":"ContainerDied","Data":"a000d8964bbe1f0ced212e818d63314b4e10b331b6c30c7db6e27decb7517580"} Sep 10 22:11:28.186326 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:28.186010 2576 scope.go:117] "RemoveContainer" containerID="a6e6edf40806d580f1c5da1af33b026fff1c7aff1205cfef6709df4d6e014772" Sep 10 22:11:28.195601 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:28.195583 2576 scope.go:117] "RemoveContainer" containerID="a6e6edf40806d580f1c5da1af33b026fff1c7aff1205cfef6709df4d6e014772" Sep 10 22:11:28.195965 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:11:28.195945 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6e6edf40806d580f1c5da1af33b026fff1c7aff1205cfef6709df4d6e014772\": container with ID starting with a6e6edf40806d580f1c5da1af33b026fff1c7aff1205cfef6709df4d6e014772 not found: ID does not exist" containerID="a6e6edf40806d580f1c5da1af33b026fff1c7aff1205cfef6709df4d6e014772" Sep 10 22:11:28.196023 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:28.195975 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6e6edf40806d580f1c5da1af33b026fff1c7aff1205cfef6709df4d6e014772"} err="failed to get container status \"a6e6edf40806d580f1c5da1af33b026fff1c7aff1205cfef6709df4d6e014772\": rpc error: code = NotFound desc = could not find container \"a6e6edf40806d580f1c5da1af33b026fff1c7aff1205cfef6709df4d6e014772\": container with ID starting with a6e6edf40806d580f1c5da1af33b026fff1c7aff1205cfef6709df4d6e014772 not found: ID does not exist" Sep 10 22:11:28.206787 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:28.206748 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7769777fc8-kk5x8"] Sep 10 22:11:28.210827 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:28.210782 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7769777fc8-kk5x8"] Sep 10 22:11:28.431030 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:28.430997 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbb1c431-8faa-4b8e-8ded-2ea86f42fbec" path="/var/lib/kubelet/pods/fbb1c431-8faa-4b8e-8ded-2ea86f42fbec/volumes" Sep 10 22:11:47.566419 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:47.566343 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:47.665570 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:47.665533 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:49.007610 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.007576 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:49.111180 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.111144 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:49.155934 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.155896 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq"] Sep 10 22:11:49.156333 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.156315 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbb1c431-8faa-4b8e-8ded-2ea86f42fbec" containerName="manager" Sep 10 22:11:49.156426 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.156335 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbb1c431-8faa-4b8e-8ded-2ea86f42fbec" containerName="manager" Sep 10 22:11:49.156426 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.156422 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbb1c431-8faa-4b8e-8ded-2ea86f42fbec" containerName="manager" Sep 10 22:11:49.161333 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.161308 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:49.165057 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.165029 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 10 22:11:49.165561 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.165539 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 10 22:11:49.165670 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.165543 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 10 22:11:49.165670 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.165576 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-spztp\"" Sep 10 22:11:49.165670 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.165606 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 22:11:49.172994 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.172962 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq"] Sep 10 22:11:49.221641 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.221600 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4562dca2-c775-43a3-b449-3344e2acaf7d-maas-api-tls\") pod \"maas-api-7f7f8945f9-88khq\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:49.221858 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.221757 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljvqc\" (UniqueName: \"kubernetes.io/projected/4562dca2-c775-43a3-b449-3344e2acaf7d-kube-api-access-ljvqc\") pod \"maas-api-7f7f8945f9-88khq\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:49.221858 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.221824 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4562dca2-c775-43a3-b449-3344e2acaf7d-openshift-service-ca\") pod \"maas-api-7f7f8945f9-88khq\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:49.221858 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.221853 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4562dca2-c775-43a3-b449-3344e2acaf7d-metrics-certs\") pod \"maas-api-7f7f8945f9-88khq\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:49.293606 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.293501 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:49.322378 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.322347 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4562dca2-c775-43a3-b449-3344e2acaf7d-maas-api-tls\") pod \"maas-api-7f7f8945f9-88khq\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:49.322549 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.322409 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljvqc\" (UniqueName: \"kubernetes.io/projected/4562dca2-c775-43a3-b449-3344e2acaf7d-kube-api-access-ljvqc\") pod \"maas-api-7f7f8945f9-88khq\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:49.322549 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.322435 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4562dca2-c775-43a3-b449-3344e2acaf7d-openshift-service-ca\") pod \"maas-api-7f7f8945f9-88khq\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:49.322549 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.322453 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4562dca2-c775-43a3-b449-3344e2acaf7d-metrics-certs\") pod \"maas-api-7f7f8945f9-88khq\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:49.322670 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:11:49.322558 2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found Sep 10 22:11:49.322670 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:11:49.322620 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4562dca2-c775-43a3-b449-3344e2acaf7d-metrics-certs podName:4562dca2-c775-43a3-b449-3344e2acaf7d nodeName:}" failed. No retries permitted until 2026-09-10 22:11:49.822603789 +0000 UTC m=+639.944302548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4562dca2-c775-43a3-b449-3344e2acaf7d-metrics-certs") pod "maas-api-7f7f8945f9-88khq" (UID: "4562dca2-c775-43a3-b449-3344e2acaf7d") : secret "maas-api-metrics-tls" not found Sep 10 22:11:49.323191 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.323166 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4562dca2-c775-43a3-b449-3344e2acaf7d-openshift-service-ca\") pod \"maas-api-7f7f8945f9-88khq\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:49.324880 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.324848 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4562dca2-c775-43a3-b449-3344e2acaf7d-maas-api-tls\") pod \"maas-api-7f7f8945f9-88khq\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:49.332691 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.332662 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljvqc\" (UniqueName: \"kubernetes.io/projected/4562dca2-c775-43a3-b449-3344e2acaf7d-kube-api-access-ljvqc\") pod \"maas-api-7f7f8945f9-88khq\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:49.494011 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.493972 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:49.660896 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.660861 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:49.753965 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.753931 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:49.827840 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.827784 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4562dca2-c775-43a3-b449-3344e2acaf7d-metrics-certs\") pod \"maas-api-7f7f8945f9-88khq\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:49.829957 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.829928 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6"] Sep 10 22:11:49.830572 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.830547 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4562dca2-c775-43a3-b449-3344e2acaf7d-metrics-certs\") pod \"maas-api-7f7f8945f9-88khq\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:49.832393 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.832379 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6" Sep 10 22:11:49.834950 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.834923 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 10 22:11:49.835190 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.835174 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jcth5\"" Sep 10 22:11:49.841648 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.841621 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6"] Sep 10 22:11:49.929498 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.929403 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqjpn\" (UniqueName: \"kubernetes.io/projected/a8c4d41b-f465-41a7-829b-19e4f958a1df-kube-api-access-bqjpn\") pod \"payload-pre-processing-59fbc47bf8-dbps6\" (UID: \"a8c4d41b-f465-41a7-829b-19e4f958a1df\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6" Sep 10 22:11:49.929498 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:49.929453 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8c4d41b-f465-41a7-829b-19e4f958a1df-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-dbps6\" (UID: \"a8c4d41b-f465-41a7-829b-19e4f958a1df\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6" Sep 10 22:11:50.030695 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:50.030654 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8c4d41b-f465-41a7-829b-19e4f958a1df-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-dbps6\" (UID: \"a8c4d41b-f465-41a7-829b-19e4f958a1df\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6" Sep 10 22:11:50.031129 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:50.030814 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqjpn\" (UniqueName: \"kubernetes.io/projected/a8c4d41b-f465-41a7-829b-19e4f958a1df-kube-api-access-bqjpn\") pod \"payload-pre-processing-59fbc47bf8-dbps6\" (UID: \"a8c4d41b-f465-41a7-829b-19e4f958a1df\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6" Sep 10 22:11:50.031417 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:50.031398 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8c4d41b-f465-41a7-829b-19e4f958a1df-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-dbps6\" (UID: \"a8c4d41b-f465-41a7-829b-19e4f958a1df\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6" Sep 10 22:11:50.039073 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:50.039047 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqjpn\" (UniqueName: \"kubernetes.io/projected/a8c4d41b-f465-41a7-829b-19e4f958a1df-kube-api-access-bqjpn\") pod \"payload-pre-processing-59fbc47bf8-dbps6\" (UID: \"a8c4d41b-f465-41a7-829b-19e4f958a1df\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6" Sep 10 22:11:50.073342 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:50.073307 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:50.149240 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:50.149210 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6" Sep 10 22:11:50.349031 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:50.348847 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq"] Sep 10 22:11:50.352062 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:11:50.352028 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4562dca2_c775_43a3_b449_3344e2acaf7d.slice/crio-cef6c03a3ba168ace03480983c846525772d19e023ae45e076b25d96d8e37f89 WatchSource:0}: Error finding container cef6c03a3ba168ace03480983c846525772d19e023ae45e076b25d96d8e37f89: Status 404 returned error can't find the container with id cef6c03a3ba168ace03480983c846525772d19e023ae45e076b25d96d8e37f89 Sep 10 22:11:50.432631 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:11:50.432594 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c4d41b_f465_41a7_829b_19e4f958a1df.slice/crio-14e6ffaf571edbe2cbaef04e1f2a5114d9f1eb7a2c4cda405e7d3d4d52eb973d WatchSource:0}: Error finding container 14e6ffaf571edbe2cbaef04e1f2a5114d9f1eb7a2c4cda405e7d3d4d52eb973d: Status 404 returned error can't find the container with id 14e6ffaf571edbe2cbaef04e1f2a5114d9f1eb7a2c4cda405e7d3d4d52eb973d Sep 10 22:11:50.433992 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:50.433857 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6"] Sep 10 22:11:51.298905 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:51.298857 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" event={"ID":"4562dca2-c775-43a3-b449-3344e2acaf7d","Type":"ContainerStarted","Data":"cef6c03a3ba168ace03480983c846525772d19e023ae45e076b25d96d8e37f89"} Sep 10 22:11:51.301174 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:51.301139 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6" event={"ID":"a8c4d41b-f465-41a7-829b-19e4f958a1df","Type":"ContainerStarted","Data":"14e6ffaf571edbe2cbaef04e1f2a5114d9f1eb7a2c4cda405e7d3d4d52eb973d"} Sep 10 22:11:51.765827 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:51.765698 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:11:53.312149 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:53.312114 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" event={"ID":"4562dca2-c775-43a3-b449-3344e2acaf7d","Type":"ContainerStarted","Data":"a0bb9bf4a2c64ea5c16b4207f1a82cf77ca617667f22f015ff2a9600ec4f8567"} Sep 10 22:11:53.312723 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:53.312202 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:11:53.313810 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:53.313768 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6" event={"ID":"a8c4d41b-f465-41a7-829b-19e4f958a1df","Type":"ContainerStarted","Data":"5f9e4b22fc8c9db6733a0e5a2dd3ce2e70c7eb20b19f8e5772e1285c78807dab"} Sep 10 22:11:53.313934 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:53.313860 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6" Sep 10 22:11:53.331548 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:53.331483 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" podStartSLOduration=2.369256304 podStartE2EDuration="4.331459853s" totalImagesPullingTime="1.962203549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:11:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:11:53.328955628 +0000 UTC m=+643.450654410" watchObservedRunningTime="2026-09-10 22:11:53.331459853 +0000 UTC m=+643.453158641" Sep 10 22:11:53.344853 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:53.344776 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6" podStartSLOduration=2.458491293 podStartE2EDuration="4.344759094s" totalImagesPullingTime="1.886267801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:11:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:11:53.343455075 +0000 UTC m=+643.465153858" watchObservedRunningTime="2026-09-10 22:11:53.344759094 +0000 UTC m=+643.466457875" Sep 10 22:11:59.324752 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:11:59.324723 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:12:00.329119 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:00.329087 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:12:00.430175 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:00.430148 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:12:04.321660 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:04.321629 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dbps6" Sep 10 22:12:11.550961 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:11.550924 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:12:18.786425 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:18.786386 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:12:32.098294 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.098256 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp"] Sep 10 22:12:32.103861 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.103833 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.106130 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.106109 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 22:12:32.106679 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.106660 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 22:12:32.106734 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.106660 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Sep 10 22:12:32.106734 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.106725 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2s65g\"" Sep 10 22:12:32.113034 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.113007 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp"] Sep 10 22:12:32.201517 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.201484 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbw9b\" (UniqueName: \"kubernetes.io/projected/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-kube-api-access-dbw9b\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.201688 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.201531 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.201688 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.201569 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.201688 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.201593 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.201688 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.201616 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.201864 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.201679 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.208204 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.208148 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:12:32.302854 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.302813 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbw9b\" (UniqueName: \"kubernetes.io/projected/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-kube-api-access-dbw9b\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.303071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.302900 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.303071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.302947 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.303071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.302987 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.303071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.303022 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.303071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.303057 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.303446 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.303425 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.303528 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.303434 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.303774 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.303751 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.305733 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.305707 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.306322 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.306296 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.312158 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.312132 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbw9b\" (UniqueName: \"kubernetes.io/projected/8ebcd5fb-6401-4b70-a840-e2fdf73d9c57-kube-api-access-dbw9b\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp\" (UID: \"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.415230 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.415135 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:32.699882 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.699783 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp"] Sep 10 22:12:32.702334 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:12:32.702305 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ebcd5fb_6401_4b70_a840_e2fdf73d9c57.slice/crio-6d8e8f05ea3ad6f256c4acb9a81001d72e33c4dce1872d7990930726edfaf844 WatchSource:0}: Error finding container 6d8e8f05ea3ad6f256c4acb9a81001d72e33c4dce1872d7990930726edfaf844: Status 404 returned error can't find the container with id 6d8e8f05ea3ad6f256c4acb9a81001d72e33c4dce1872d7990930726edfaf844 Sep 10 22:12:32.703724 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:32.703702 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 22:12:33.514631 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:33.514587 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" event={"ID":"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57","Type":"ContainerStarted","Data":"6d8e8f05ea3ad6f256c4acb9a81001d72e33c4dce1872d7990930726edfaf844"} Sep 10 22:12:34.520703 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:34.520671 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" event={"ID":"8ebcd5fb-6401-4b70-a840-e2fdf73d9c57","Type":"ContainerStarted","Data":"0de1df8c6a8967228e4c09b9a0208ad9a3e2cde3598bb69e68a1d501a7533f5e"} Sep 10 22:12:34.540824 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:34.540752 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" podStartSLOduration=1.560608717 podStartE2EDuration="2.540732177s" totalImagesPullingTime="980.12346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:12:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:12:34.538417196 +0000 UTC m=+684.660115977" watchObservedRunningTime="2026-09-10 22:12:34.540732177 +0000 UTC m=+684.662430993" Sep 10 22:12:40.203466 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.203423 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l"] Sep 10 22:12:40.207178 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.207153 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.209384 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.209352 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 10 22:12:40.217127 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.217097 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l"] Sep 10 22:12:40.242576 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.242543 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:12:40.278316 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.278283 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbddc39e-31bd-4631-ac51-36cd587232c2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.278316 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.278315 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbddc39e-31bd-4631-ac51-36cd587232c2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.278506 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.278338 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stbmd\" (UniqueName: \"kubernetes.io/projected/fbddc39e-31bd-4631-ac51-36cd587232c2-kube-api-access-stbmd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.278506 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.278363 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbddc39e-31bd-4631-ac51-36cd587232c2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.278506 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.278414 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbddc39e-31bd-4631-ac51-36cd587232c2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.278506 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.278432 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbddc39e-31bd-4631-ac51-36cd587232c2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.379496 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.379439 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbddc39e-31bd-4631-ac51-36cd587232c2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.379496 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.379513 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbddc39e-31bd-4631-ac51-36cd587232c2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.379721 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.379544 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stbmd\" (UniqueName: \"kubernetes.io/projected/fbddc39e-31bd-4631-ac51-36cd587232c2-kube-api-access-stbmd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.379721 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.379573 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbddc39e-31bd-4631-ac51-36cd587232c2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.379721 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.379617 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbddc39e-31bd-4631-ac51-36cd587232c2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.379721 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.379642 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbddc39e-31bd-4631-ac51-36cd587232c2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.380051 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.380025 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbddc39e-31bd-4631-ac51-36cd587232c2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.380051 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.380042 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbddc39e-31bd-4631-ac51-36cd587232c2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.380172 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.380085 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbddc39e-31bd-4631-ac51-36cd587232c2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.381770 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.381750 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbddc39e-31bd-4631-ac51-36cd587232c2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.382146 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.382124 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbddc39e-31bd-4631-ac51-36cd587232c2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.388623 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.388595 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stbmd\" (UniqueName: \"kubernetes.io/projected/fbddc39e-31bd-4631-ac51-36cd587232c2-kube-api-access-stbmd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4t57l\" (UID: \"fbddc39e-31bd-4631-ac51-36cd587232c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.518584 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.518485 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:40.793339 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:40.793318 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l"] Sep 10 22:12:40.795638 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:12:40.795609 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbddc39e_31bd_4631_ac51_36cd587232c2.slice/crio-32278fe446309bd65472aca0035c66fe6dbef45b4ea8e5c1b79cd895986947e6 WatchSource:0}: Error finding container 32278fe446309bd65472aca0035c66fe6dbef45b4ea8e5c1b79cd895986947e6: Status 404 returned error can't find the container with id 32278fe446309bd65472aca0035c66fe6dbef45b4ea8e5c1b79cd895986947e6 Sep 10 22:12:41.556090 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:41.556052 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" event={"ID":"fbddc39e-31bd-4631-ac51-36cd587232c2","Type":"ContainerStarted","Data":"b16b704d6d7fc0d2d6016c3d791ebd8c840190a6a652d5acbea70d8293673f64"} Sep 10 22:12:41.556485 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:41.556103 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" event={"ID":"fbddc39e-31bd-4631-ac51-36cd587232c2","Type":"ContainerStarted","Data":"32278fe446309bd65472aca0035c66fe6dbef45b4ea8e5c1b79cd895986947e6"} Sep 10 22:12:41.574580 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:41.574501 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" podStartSLOduration=1.329820568 podStartE2EDuration="1.574481222s" totalImagesPullingTime="244.660654ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:12:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:12:41.572791959 +0000 UTC m=+691.694490742" watchObservedRunningTime="2026-09-10 22:12:41.574481222 +0000 UTC m=+691.696180061" Sep 10 22:12:42.415824 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:42.415758 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:42.416021 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:42.415870 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:42.431614 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:42.431580 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:42.572830 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:42.572779 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp" Sep 10 22:12:48.720484 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:48.720426 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:12:50.518865 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:50.518827 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:50.519281 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:50.518895 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:50.532214 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:50.532176 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:50.610669 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:50.610640 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4t57l" Sep 10 22:12:57.626473 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.626437 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx"] Sep 10 22:12:57.630235 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.630202 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.633860 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.633832 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 10 22:12:57.644665 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.644629 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.644877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.644684 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz246\" (UniqueName: \"kubernetes.io/projected/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-kube-api-access-bz246\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.644877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.644720 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.644877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.644787 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.644877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.644856 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.645103 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.644904 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.646320 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.646294 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx"] Sep 10 22:12:57.699155 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.699124 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:12:57.746202 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.746139 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.746409 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.746269 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.746409 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.746315 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz246\" (UniqueName: \"kubernetes.io/projected/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-kube-api-access-bz246\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.746409 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.746344 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.746409 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.746383 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.746409 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.746409 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.746687 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.746662 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.746997 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.746966 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.747116 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.746999 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.748750 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.748732 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.749162 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.749142 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.755138 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.755108 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz246\" (UniqueName: \"kubernetes.io/projected/d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a-kube-api-access-bz246\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bhjwx\" (UID: \"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:57.941506 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:57.941414 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:12:58.215976 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:58.215889 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx"] Sep 10 22:12:58.219536 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:12:58.219500 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3a1cfb4_9309_422b_b9be_9ee4b7e0e30a.slice/crio-15ff7320236da24fc074e54191d3857b2c0bcf763eb4863376fbd780879c84d6 WatchSource:0}: Error finding container 15ff7320236da24fc074e54191d3857b2c0bcf763eb4863376fbd780879c84d6: Status 404 returned error can't find the container with id 15ff7320236da24fc074e54191d3857b2c0bcf763eb4863376fbd780879c84d6 Sep 10 22:12:58.640788 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:58.640746 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" event={"ID":"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a","Type":"ContainerStarted","Data":"71a42e71ac485ee55e5729c3ddc05eac219ebc8e932b6fa7f68f80d41f344414"} Sep 10 22:12:58.641206 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:58.640847 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" event={"ID":"d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a","Type":"ContainerStarted","Data":"15ff7320236da24fc074e54191d3857b2c0bcf763eb4863376fbd780879c84d6"} Sep 10 22:12:58.659754 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:12:58.659698 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" podStartSLOduration=1.47260739 podStartE2EDuration="1.659681293s" totalImagesPullingTime="187.073903ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:12:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:12:58.658525435 +0000 UTC m=+708.780224244" watchObservedRunningTime="2026-09-10 22:12:58.659681293 +0000 UTC m=+708.781380075" Sep 10 22:13:02.640576 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:13:02.640464 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:13:07.941625 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:13:07.941587 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:13:07.942146 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:13:07.941657 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:13:07.954259 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:13:07.954229 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:13:08.706192 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:13:08.706160 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bhjwx" Sep 10 22:13:15.793106 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:13:15.793070 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:13:57.792465 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:13:57.792426 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:18.384203 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:18.384145 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:21.431936 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:21.431868 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:21.540878 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:21.540840 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:22.368084 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:22.368052 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:22.471436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:22.471382 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:23.414165 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:23.414116 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:23.535986 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:23.535950 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:38.766285 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:38.766186 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:38.884711 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:38.884672 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:39.123881 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:39.123845 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:39.381245 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:39.381122 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:40.206862 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:40.206825 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:40.350170 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:40.350138 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:40.672689 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:40.672658 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:41.854637 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:41.854598 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:41.941902 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:41.941861 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:43.138221 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.138189 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7"] Sep 10 22:14:43.140713 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.140663 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:43.144046 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.143999 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-e5a2d7-serving-cert\"" Sep 10 22:14:43.156307 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.156276 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7"] Sep 10 22:14:43.178613 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.178585 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:43.251132 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.251090 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72feb2e1-36b7-4a0f-b883-5f238cedecf5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:43.251325 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.251160 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72feb2e1-36b7-4a0f-b883-5f238cedecf5-metrics-certs\") pod \"maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:43.251325 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.251192 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnhbh\" (UniqueName: \"kubernetes.io/projected/72feb2e1-36b7-4a0f-b883-5f238cedecf5-kube-api-access-jnhbh\") pod \"maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:43.251325 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.251220 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72feb2e1-36b7-4a0f-b883-5f238cedecf5-maas-api-tls\") pod \"maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:43.294065 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.294026 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:43.352097 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.352056 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72feb2e1-36b7-4a0f-b883-5f238cedecf5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:43.352289 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.352193 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72feb2e1-36b7-4a0f-b883-5f238cedecf5-metrics-certs\") pod \"maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:43.352289 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.352225 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnhbh\" (UniqueName: \"kubernetes.io/projected/72feb2e1-36b7-4a0f-b883-5f238cedecf5-kube-api-access-jnhbh\") pod \"maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:43.352289 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.352244 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72feb2e1-36b7-4a0f-b883-5f238cedecf5-maas-api-tls\") pod \"maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:43.352462 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:14:43.352413 2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-serving-cert: secret "maas-api-e2e-inf-a-e5a2d7-serving-cert" not found Sep 10 22:14:43.352516 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:14:43.352471 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72feb2e1-36b7-4a0f-b883-5f238cedecf5-maas-api-tls podName:72feb2e1-36b7-4a0f-b883-5f238cedecf5 nodeName:}" failed. No retries permitted until 2026-09-10 22:14:43.852455545 +0000 UTC m=+813.974154305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/72feb2e1-36b7-4a0f-b883-5f238cedecf5-maas-api-tls") pod "maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" (UID: "72feb2e1-36b7-4a0f-b883-5f238cedecf5") : secret "maas-api-e2e-inf-a-e5a2d7-serving-cert" not found Sep 10 22:14:43.352753 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.352731 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72feb2e1-36b7-4a0f-b883-5f238cedecf5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:43.354790 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.354764 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72feb2e1-36b7-4a0f-b883-5f238cedecf5-metrics-certs\") pod \"maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:43.371867 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.371832 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnhbh\" (UniqueName: \"kubernetes.io/projected/72feb2e1-36b7-4a0f-b883-5f238cedecf5-kube-api-access-jnhbh\") pod \"maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:43.722432 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.722397 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:43.838909 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.838875 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:43.856944 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.856910 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72feb2e1-36b7-4a0f-b883-5f238cedecf5-maas-api-tls\") pod \"maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:43.859520 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.859490 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72feb2e1-36b7-4a0f-b883-5f238cedecf5-maas-api-tls\") pod \"maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:43.939119 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.939076 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:43.980346 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.976088 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf"] Sep 10 22:14:43.981933 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.981890 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" Sep 10 22:14:43.984125 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.984089 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-e5a2d7\"" Sep 10 22:14:43.984563 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.984541 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-e5a2d7-dockercfg-4vx96\"" Sep 10 22:14:43.994194 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:43.994164 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf"] Sep 10 22:14:44.022345 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.022309 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w"] Sep 10 22:14:44.024994 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.024974 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" Sep 10 22:14:44.037498 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.037463 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w"] Sep 10 22:14:44.052505 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.052464 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:44.059191 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.059160 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89spv\" (UniqueName: \"kubernetes.io/projected/cb47e7b6-331c-41c5-8b24-3c1533b7fd09-kube-api-access-89spv\") pod \"payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf\" (UID: \"cb47e7b6-331c-41c5-8b24-3c1533b7fd09\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" Sep 10 22:14:44.059351 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.059238 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb47e7b6-331c-41c5-8b24-3c1533b7fd09-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf\" (UID: \"cb47e7b6-331c-41c5-8b24-3c1533b7fd09\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" Sep 10 22:14:44.160364 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.160316 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb47e7b6-331c-41c5-8b24-3c1533b7fd09-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf\" (UID: \"cb47e7b6-331c-41c5-8b24-3c1533b7fd09\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" Sep 10 22:14:44.160878 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.160434 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20953f3c-7e42-4683-8f19-ef7379cd2c04-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w\" (UID: \"20953f3c-7e42-4683-8f19-ef7379cd2c04\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" Sep 10 22:14:44.160878 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.160473 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89spv\" (UniqueName: \"kubernetes.io/projected/cb47e7b6-331c-41c5-8b24-3c1533b7fd09-kube-api-access-89spv\") pod \"payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf\" (UID: \"cb47e7b6-331c-41c5-8b24-3c1533b7fd09\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" Sep 10 22:14:44.160878 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.160575 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd24h\" (UniqueName: \"kubernetes.io/projected/20953f3c-7e42-4683-8f19-ef7379cd2c04-kube-api-access-gd24h\") pod \"payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w\" (UID: \"20953f3c-7e42-4683-8f19-ef7379cd2c04\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" Sep 10 22:14:44.161265 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.161243 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb47e7b6-331c-41c5-8b24-3c1533b7fd09-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf\" (UID: \"cb47e7b6-331c-41c5-8b24-3c1533b7fd09\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" Sep 10 22:14:44.168481 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.168451 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89spv\" (UniqueName: \"kubernetes.io/projected/cb47e7b6-331c-41c5-8b24-3c1533b7fd09-kube-api-access-89spv\") pod \"payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf\" (UID: \"cb47e7b6-331c-41c5-8b24-3c1533b7fd09\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" Sep 10 22:14:44.262003 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.261909 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd24h\" (UniqueName: \"kubernetes.io/projected/20953f3c-7e42-4683-8f19-ef7379cd2c04-kube-api-access-gd24h\") pod \"payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w\" (UID: \"20953f3c-7e42-4683-8f19-ef7379cd2c04\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" Sep 10 22:14:44.262173 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.262036 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20953f3c-7e42-4683-8f19-ef7379cd2c04-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w\" (UID: \"20953f3c-7e42-4683-8f19-ef7379cd2c04\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" Sep 10 22:14:44.262737 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.262717 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20953f3c-7e42-4683-8f19-ef7379cd2c04-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w\" (UID: \"20953f3c-7e42-4683-8f19-ef7379cd2c04\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" Sep 10 22:14:44.270366 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.270328 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd24h\" (UniqueName: \"kubernetes.io/projected/20953f3c-7e42-4683-8f19-ef7379cd2c04-kube-api-access-gd24h\") pod \"payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w\" (UID: \"20953f3c-7e42-4683-8f19-ef7379cd2c04\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" Sep 10 22:14:44.294464 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.294430 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" Sep 10 22:14:44.333179 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.331690 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7"] Sep 10 22:14:44.335651 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:14:44.335622 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72feb2e1_36b7_4a0f_b883_5f238cedecf5.slice/crio-0abc07dfcfb0dfeca27ccb604f6e12a013b1764d986d111a2dabf686fba0c673 WatchSource:0}: Error finding container 0abc07dfcfb0dfeca27ccb604f6e12a013b1764d986d111a2dabf686fba0c673: Status 404 returned error can't find the container with id 0abc07dfcfb0dfeca27ccb604f6e12a013b1764d986d111a2dabf686fba0c673 Sep 10 22:14:44.336711 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.336692 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" Sep 10 22:14:44.575375 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.575348 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf"] Sep 10 22:14:44.577304 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:14:44.577277 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb47e7b6_331c_41c5_8b24_3c1533b7fd09.slice/crio-701253d63f19b978e7f962003714a58a3907932e20a65bcf66b46b212f8b27c7 WatchSource:0}: Error finding container 701253d63f19b978e7f962003714a58a3907932e20a65bcf66b46b212f8b27c7: Status 404 returned error can't find the container with id 701253d63f19b978e7f962003714a58a3907932e20a65bcf66b46b212f8b27c7 Sep 10 22:14:44.625555 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:44.625533 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w"] Sep 10 22:14:44.631033 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:14:44.631004 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20953f3c_7e42_4683_8f19_ef7379cd2c04.slice/crio-d17df422a6b4f6a8467050029c0c76259054edc725815434d9989f835fc4935e WatchSource:0}: Error finding container d17df422a6b4f6a8467050029c0c76259054edc725815434d9989f835fc4935e: Status 404 returned error can't find the container with id d17df422a6b4f6a8467050029c0c76259054edc725815434d9989f835fc4935e Sep 10 22:14:45.166564 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:45.166531 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" event={"ID":"72feb2e1-36b7-4a0f-b883-5f238cedecf5","Type":"ContainerStarted","Data":"ece66bcbef81e39825ee20c880b7c3e76aca39f883a040edaf4542ea5b76e171"} Sep 10 22:14:45.167091 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:45.166582 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" event={"ID":"72feb2e1-36b7-4a0f-b883-5f238cedecf5","Type":"ContainerStarted","Data":"0abc07dfcfb0dfeca27ccb604f6e12a013b1764d986d111a2dabf686fba0c673"} Sep 10 22:14:45.167091 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:45.166691 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:45.168359 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:45.168334 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" event={"ID":"cb47e7b6-331c-41c5-8b24-3c1533b7fd09","Type":"ContainerStarted","Data":"9c1ec94a143bd0f17357b64cab252fe94b759c82b629f42e51e47dc257ba031b"} Sep 10 22:14:45.168484 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:45.168389 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" Sep 10 22:14:45.168484 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:45.168416 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" event={"ID":"cb47e7b6-331c-41c5-8b24-3c1533b7fd09","Type":"ContainerStarted","Data":"701253d63f19b978e7f962003714a58a3907932e20a65bcf66b46b212f8b27c7"} Sep 10 22:14:45.169890 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:45.169871 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" event={"ID":"20953f3c-7e42-4683-8f19-ef7379cd2c04","Type":"ContainerStarted","Data":"dee558c9363faf7800fd4e46444d6dc74cc2623cc7aeebf0a489cf3b95d912a7"} Sep 10 22:14:45.170005 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:45.169907 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" event={"ID":"20953f3c-7e42-4683-8f19-ef7379cd2c04","Type":"ContainerStarted","Data":"d17df422a6b4f6a8467050029c0c76259054edc725815434d9989f835fc4935e"} Sep 10 22:14:45.170061 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:45.170051 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" Sep 10 22:14:45.184831 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:45.184762 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" podStartSLOduration=1.784948185 podStartE2EDuration="2.184742049s" totalImagesPullingTime="399.793864ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:14:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:14:45.183012537 +0000 UTC m=+815.304711324" watchObservedRunningTime="2026-09-10 22:14:45.184742049 +0000 UTC m=+815.306440833" Sep 10 22:14:45.200792 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:45.200727 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" podStartSLOduration=2.200705069 podStartE2EDuration="2.200705069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:14:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:14:45.197872555 +0000 UTC m=+815.319571339" watchObservedRunningTime="2026-09-10 22:14:45.200705069 +0000 UTC m=+815.322403852" Sep 10 22:14:45.215467 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:45.215403 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" podStartSLOduration=2.215378723 podStartE2EDuration="2.215378723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:14:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:14:45.213232097 +0000 UTC m=+815.334930880" watchObservedRunningTime="2026-09-10 22:14:45.215378723 +0000 UTC m=+815.337077506" Sep 10 22:14:45.901107 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:45.901069 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:46.194048 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:46.193913 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:46.459292 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:46.459179 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:46.740902 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:46.740774 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:46.838432 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:46.838395 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:46.951085 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:46.951051 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:47.442449 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:47.442417 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:47.901303 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:47.901268 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:48.767622 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:48.767590 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:48.865247 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:48.865210 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:49.900233 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:49.900195 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw"] Sep 10 22:14:49.904208 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:49.904183 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:49.907587 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:49.907560 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-d6a4de-serving-cert\"" Sep 10 22:14:49.920042 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:49.920014 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw"] Sep 10 22:14:49.950657 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:49.950625 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:50.019535 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.019502 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa96df94-e8c7-4904-99a0-811cccb98aa0-metrics-certs\") pod \"maas-api-e2e-autores-d6a4de-5978677b7-4p2hw\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:50.019707 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.019548 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdh2n\" (UniqueName: \"kubernetes.io/projected/aa96df94-e8c7-4904-99a0-811cccb98aa0-kube-api-access-kdh2n\") pod \"maas-api-e2e-autores-d6a4de-5978677b7-4p2hw\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:50.019707 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.019644 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa96df94-e8c7-4904-99a0-811cccb98aa0-maas-api-tls\") pod \"maas-api-e2e-autores-d6a4de-5978677b7-4p2hw\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:50.019707 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.019674 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa96df94-e8c7-4904-99a0-811cccb98aa0-openshift-service-ca\") pod \"maas-api-e2e-autores-d6a4de-5978677b7-4p2hw\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:50.054041 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.054007 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:50.120350 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.120312 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa96df94-e8c7-4904-99a0-811cccb98aa0-metrics-certs\") pod \"maas-api-e2e-autores-d6a4de-5978677b7-4p2hw\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:50.120530 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.120356 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdh2n\" (UniqueName: \"kubernetes.io/projected/aa96df94-e8c7-4904-99a0-811cccb98aa0-kube-api-access-kdh2n\") pod \"maas-api-e2e-autores-d6a4de-5978677b7-4p2hw\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:50.120530 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.120408 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa96df94-e8c7-4904-99a0-811cccb98aa0-maas-api-tls\") pod \"maas-api-e2e-autores-d6a4de-5978677b7-4p2hw\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:50.120530 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.120444 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa96df94-e8c7-4904-99a0-811cccb98aa0-openshift-service-ca\") pod \"maas-api-e2e-autores-d6a4de-5978677b7-4p2hw\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:50.120638 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:14:50.120556 2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-serving-cert: secret "maas-api-e2e-autores-d6a4de-serving-cert" not found Sep 10 22:14:50.120638 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:14:50.120634 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa96df94-e8c7-4904-99a0-811cccb98aa0-maas-api-tls podName:aa96df94-e8c7-4904-99a0-811cccb98aa0 nodeName:}" failed. No retries permitted until 2026-09-10 22:14:50.620609533 +0000 UTC m=+820.742308295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aa96df94-e8c7-4904-99a0-811cccb98aa0-maas-api-tls") pod "maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" (UID: "aa96df94-e8c7-4904-99a0-811cccb98aa0") : secret "maas-api-e2e-autores-d6a4de-serving-cert" not found Sep 10 22:14:50.121186 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.121165 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa96df94-e8c7-4904-99a0-811cccb98aa0-openshift-service-ca\") pod \"maas-api-e2e-autores-d6a4de-5978677b7-4p2hw\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:50.123034 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.123015 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa96df94-e8c7-4904-99a0-811cccb98aa0-metrics-certs\") pod \"maas-api-e2e-autores-d6a4de-5978677b7-4p2hw\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:50.131169 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.131135 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdh2n\" (UniqueName: \"kubernetes.io/projected/aa96df94-e8c7-4904-99a0-811cccb98aa0-kube-api-access-kdh2n\") pod \"maas-api-e2e-autores-d6a4de-5978677b7-4p2hw\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:50.233983 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.233899 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:50.416342 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.416305 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:50.527006 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.526914 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:50.625936 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.625752 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa96df94-e8c7-4904-99a0-811cccb98aa0-maas-api-tls\") pod \"maas-api-e2e-autores-d6a4de-5978677b7-4p2hw\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:50.629372 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.629341 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa96df94-e8c7-4904-99a0-811cccb98aa0-maas-api-tls\") pod \"maas-api-e2e-autores-d6a4de-5978677b7-4p2hw\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:50.795874 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.795776 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:50.819165 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.819129 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:50.908567 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:50.908533 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:51.079470 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.079432 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck"] Sep 10 22:14:51.082762 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.082737 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" Sep 10 22:14:51.084967 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.084939 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-d6a4de-dockercfg-s9cnc\"" Sep 10 22:14:51.084967 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.084956 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-d6a4de\"" Sep 10 22:14:51.097677 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.097642 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck"] Sep 10 22:14:51.104759 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.104711 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw"] Sep 10 22:14:51.110173 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:14:51.110147 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa96df94_e8c7_4904_99a0_811cccb98aa0.slice/crio-90d5a2a296187451d4d4231f40367cab1b85c71a095fc1461ee55b02c157ad4f WatchSource:0}: Error finding container 90d5a2a296187451d4d4231f40367cab1b85c71a095fc1461ee55b02c157ad4f: Status 404 returned error can't find the container with id 90d5a2a296187451d4d4231f40367cab1b85c71a095fc1461ee55b02c157ad4f Sep 10 22:14:51.181679 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.181650 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:14:51.205405 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.205359 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" event={"ID":"aa96df94-e8c7-4904-99a0-811cccb98aa0","Type":"ContainerStarted","Data":"90d5a2a296187451d4d4231f40367cab1b85c71a095fc1461ee55b02c157ad4f"} Sep 10 22:14:51.232970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.232936 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6c7p\" (UniqueName: \"kubernetes.io/projected/aebe8545-0d97-4c82-843c-7563fb4fb3e8-kube-api-access-r6c7p\") pod \"payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck\" (UID: \"aebe8545-0d97-4c82-843c-7563fb4fb3e8\") " pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" Sep 10 22:14:51.233178 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.233072 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aebe8545-0d97-4c82-843c-7563fb4fb3e8-plugins-config-volume\") pod \"payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck\" (UID: \"aebe8545-0d97-4c82-843c-7563fb4fb3e8\") " pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" Sep 10 22:14:51.334298 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.334213 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6c7p\" (UniqueName: \"kubernetes.io/projected/aebe8545-0d97-4c82-843c-7563fb4fb3e8-kube-api-access-r6c7p\") pod \"payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck\" (UID: \"aebe8545-0d97-4c82-843c-7563fb4fb3e8\") " pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" Sep 10 22:14:51.334461 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.334397 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aebe8545-0d97-4c82-843c-7563fb4fb3e8-plugins-config-volume\") pod \"payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck\" (UID: \"aebe8545-0d97-4c82-843c-7563fb4fb3e8\") " pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" Sep 10 22:14:51.335117 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.335096 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aebe8545-0d97-4c82-843c-7563fb4fb3e8-plugins-config-volume\") pod \"payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck\" (UID: \"aebe8545-0d97-4c82-843c-7563fb4fb3e8\") " pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" Sep 10 22:14:51.343271 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.343230 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6c7p\" (UniqueName: \"kubernetes.io/projected/aebe8545-0d97-4c82-843c-7563fb4fb3e8-kube-api-access-r6c7p\") pod \"payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck\" (UID: \"aebe8545-0d97-4c82-843c-7563fb4fb3e8\") " pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" Sep 10 22:14:51.395446 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.395401 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" Sep 10 22:14:51.676079 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:51.675996 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck"] Sep 10 22:14:51.679123 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:14:51.679092 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaebe8545_0d97_4c82_843c_7563fb4fb3e8.slice/crio-7cb264d5335bb936efa3316b632f6d34472fb05c8c7cf1a89769de7e9b797a6b WatchSource:0}: Error finding container 7cb264d5335bb936efa3316b632f6d34472fb05c8c7cf1a89769de7e9b797a6b: Status 404 returned error can't find the container with id 7cb264d5335bb936efa3316b632f6d34472fb05c8c7cf1a89769de7e9b797a6b Sep 10 22:14:52.100964 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:52.100918 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:52.200883 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:52.200841 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:52.211880 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:52.211846 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" event={"ID":"aa96df94-e8c7-4904-99a0-811cccb98aa0","Type":"ContainerStarted","Data":"7d72cdefb2d1c0e18fd3382d6b63c56047eaa10da2dad2308d75258b22821971"} Sep 10 22:14:52.212083 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:52.211957 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:14:52.213653 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:52.213623 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" event={"ID":"aebe8545-0d97-4c82-843c-7563fb4fb3e8","Type":"ContainerStarted","Data":"49d05203fc041e5b8fbcfc2596dce8849c9ec1f08c32ba05b41cec19127eef35"} Sep 10 22:14:52.213784 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:52.213671 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" event={"ID":"aebe8545-0d97-4c82-843c-7563fb4fb3e8","Type":"ContainerStarted","Data":"7cb264d5335bb936efa3316b632f6d34472fb05c8c7cf1a89769de7e9b797a6b"} Sep 10 22:14:52.213784 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:52.213713 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" Sep 10 22:14:52.231399 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:52.231338 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" podStartSLOduration=2.8768681000000003 podStartE2EDuration="3.231319446s" totalImagesPullingTime="354.451346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:14:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:14:52.230484925 +0000 UTC m=+822.352183735" watchObservedRunningTime="2026-09-10 22:14:52.231319446 +0000 UTC m=+822.353018228" Sep 10 22:14:52.251110 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:52.251039 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" podStartSLOduration=1.251013653 podStartE2EDuration="1.251013653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:14:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:14:52.249252893 +0000 UTC m=+822.370951676" watchObservedRunningTime="2026-09-10 22:14:52.251013653 +0000 UTC m=+822.372712437" Sep 10 22:14:52.588379 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:52.588348 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:52.685558 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:52.685523 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:52.771057 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:52.771028 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:56.178302 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:56.178270 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" Sep 10 22:14:56.178714 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:56.178352 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" Sep 10 22:14:57.665495 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:57.665462 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:57.834478 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:57.834415 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:57.923213 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:57.923034 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:58.023071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:58.023039 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:14:58.227077 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:14:58.227001 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:15:00.700155 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:00.700122 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:02.388017 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:02.387986 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:03.221751 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:03.221720 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" Sep 10 22:15:04.272558 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:04.272524 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:06.753028 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:06.753000 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:07.103598 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:07.103563 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:07.553223 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:07.553145 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:08.430279 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:08.430246 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:11.110756 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:11.110720 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:11.205371 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:11.205318 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:11.405567 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:11.405427 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:12.810758 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:12.810726 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:12.923128 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:12.923093 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:13.190252 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.190208 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:13.521091 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.520997 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:13.627114 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.627085 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:13.641318 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.641288 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl"] Sep 10 22:15:13.645105 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.645078 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" Sep 10 22:15:13.647621 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.647596 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-1c252b\"" Sep 10 22:15:13.647826 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.647759 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-1c252b-dockercfg-9v6sh\"" Sep 10 22:15:13.652243 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.651140 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8"] Sep 10 22:15:13.662709 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.661214 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl"] Sep 10 22:15:13.662709 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.661350 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" Sep 10 22:15:13.668547 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.668512 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8"] Sep 10 22:15:13.717975 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.717938 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:13.764884 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.764851 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gvlr\" (UniqueName: \"kubernetes.io/projected/e6dc39f0-3ceb-4527-9b17-22cc8526d6c9-kube-api-access-5gvlr\") pod \"payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl\" (UID: \"e6dc39f0-3ceb-4527-9b17-22cc8526d6c9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" Sep 10 22:15:13.765038 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.764896 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbwhv\" (UniqueName: \"kubernetes.io/projected/fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8-kube-api-access-xbwhv\") pod \"payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8\" (UID: \"fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" Sep 10 22:15:13.765038 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.764939 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8\" (UID: \"fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" Sep 10 22:15:13.765113 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.765064 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6dc39f0-3ceb-4527-9b17-22cc8526d6c9-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl\" (UID: \"e6dc39f0-3ceb-4527-9b17-22cc8526d6c9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" Sep 10 22:15:13.866487 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.866448 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gvlr\" (UniqueName: \"kubernetes.io/projected/e6dc39f0-3ceb-4527-9b17-22cc8526d6c9-kube-api-access-5gvlr\") pod \"payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl\" (UID: \"e6dc39f0-3ceb-4527-9b17-22cc8526d6c9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" Sep 10 22:15:13.866487 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.866490 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbwhv\" (UniqueName: \"kubernetes.io/projected/fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8-kube-api-access-xbwhv\") pod \"payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8\" (UID: \"fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" Sep 10 22:15:13.867017 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.866514 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8\" (UID: \"fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" Sep 10 22:15:13.867017 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.866557 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6dc39f0-3ceb-4527-9b17-22cc8526d6c9-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl\" (UID: \"e6dc39f0-3ceb-4527-9b17-22cc8526d6c9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" Sep 10 22:15:13.867330 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.867308 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8\" (UID: \"fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" Sep 10 22:15:13.867367 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.867338 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6dc39f0-3ceb-4527-9b17-22cc8526d6c9-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl\" (UID: \"e6dc39f0-3ceb-4527-9b17-22cc8526d6c9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" Sep 10 22:15:13.874496 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.874463 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gvlr\" (UniqueName: \"kubernetes.io/projected/e6dc39f0-3ceb-4527-9b17-22cc8526d6c9-kube-api-access-5gvlr\") pod \"payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl\" (UID: \"e6dc39f0-3ceb-4527-9b17-22cc8526d6c9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" Sep 10 22:15:13.874496 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.874492 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbwhv\" (UniqueName: \"kubernetes.io/projected/fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8-kube-api-access-xbwhv\") pod \"payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8\" (UID: \"fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" Sep 10 22:15:13.972720 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.972682 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" Sep 10 22:15:13.979937 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.979912 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" Sep 10 22:15:13.992293 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:13.992263 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:14.255310 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:14.255281 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl"] Sep 10 22:15:14.257507 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:15:14.257479 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6dc39f0_3ceb_4527_9b17_22cc8526d6c9.slice/crio-b3a9ea62e55d2838720a703a2b87046e50879418d5e69f2e7a05ae0ff60f7ff7 WatchSource:0}: Error finding container b3a9ea62e55d2838720a703a2b87046e50879418d5e69f2e7a05ae0ff60f7ff7: Status 404 returned error can't find the container with id b3a9ea62e55d2838720a703a2b87046e50879418d5e69f2e7a05ae0ff60f7ff7 Sep 10 22:15:14.292592 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:14.292561 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8"] Sep 10 22:15:14.296164 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:15:14.296132 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc070cc7_1ba8_4c20_a1e2_c9797f9b31a8.slice/crio-5b7c2cbf4f8cb5b352ddcedd3834e35875554a6ecc0a45f892f75b94dde1bcb5 WatchSource:0}: Error finding container 5b7c2cbf4f8cb5b352ddcedd3834e35875554a6ecc0a45f892f75b94dde1bcb5: Status 404 returned error can't find the container with id 5b7c2cbf4f8cb5b352ddcedd3834e35875554a6ecc0a45f892f75b94dde1bcb5 Sep 10 22:15:14.337826 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:14.337764 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" event={"ID":"e6dc39f0-3ceb-4527-9b17-22cc8526d6c9","Type":"ContainerStarted","Data":"b3a9ea62e55d2838720a703a2b87046e50879418d5e69f2e7a05ae0ff60f7ff7"} Sep 10 22:15:14.339129 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:14.339101 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" event={"ID":"fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8","Type":"ContainerStarted","Data":"5b7c2cbf4f8cb5b352ddcedd3834e35875554a6ecc0a45f892f75b94dde1bcb5"} Sep 10 22:15:14.934066 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:14.934032 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:15.347271 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:15.347234 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" event={"ID":"e6dc39f0-3ceb-4527-9b17-22cc8526d6c9","Type":"ContainerStarted","Data":"afb2c2d5da65cf1add80560664b59d8ea64e6647a3c7c464a48cef8a0ccb1565"} Sep 10 22:15:15.347477 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:15.347404 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" Sep 10 22:15:15.348978 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:15.348948 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" event={"ID":"fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8","Type":"ContainerStarted","Data":"d7f4926511e27d1b88cdd567f1a535f340957b37556cdf444f10989c362771da"} Sep 10 22:15:15.349107 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:15.349083 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" Sep 10 22:15:15.362538 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:15.362468 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" podStartSLOduration=2.362445345 podStartE2EDuration="2.362445345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:15:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:15:15.361835541 +0000 UTC m=+845.483534323" watchObservedRunningTime="2026-09-10 22:15:15.362445345 +0000 UTC m=+845.484144127" Sep 10 22:15:15.379041 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:15.378984 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" podStartSLOduration=2.378968001 podStartE2EDuration="2.378968001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:15:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:15:15.376327814 +0000 UTC m=+845.498026598" watchObservedRunningTime="2026-09-10 22:15:15.378968001 +0000 UTC m=+845.500666783" Sep 10 22:15:15.703418 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:15.703326 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:16.500464 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:16.500430 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:26.356566 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:26.356534 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" Sep 10 22:15:26.357068 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:26.356610 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" Sep 10 22:15:32.223686 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:32.223653 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:32.324950 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:32.324917 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:32.425191 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:32.425058 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:32.520335 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:32.520250 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:32.714394 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:32.714359 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:33.146242 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:33.146214 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:33.251773 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:33.251737 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:33.373167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:33.373128 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:33.802355 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:33.802319 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:33.923689 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:33.923660 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:34.122889 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:34.122853 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:35.983838 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:35.983794 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:36.860836 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:36.860779 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:36.977114 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:36.977083 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:37.105197 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:37.105159 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:37.214952 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:37.214856 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:37.344274 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:37.344237 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:37.936502 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:37.936472 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:38.028473 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:38.028446 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:48.569437 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:48.569393 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:48.662838 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:48.662781 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:48.772895 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:48.772861 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:55.069293 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:55.069258 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:55.239140 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:55.239107 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:55.370333 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:55.370257 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:15:55.464359 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:15:55.464325 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:04.059501 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:04.059468 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:05.217236 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:05.217201 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:05.444229 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:05.444191 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:09.062028 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:09.061998 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:09.231097 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:09.231061 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:09.417542 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:09.417511 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:09.626938 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:09.626875 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:09.912873 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:09.912840 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:10.032483 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:10.032454 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:10.335584 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:10.335555 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:10.764559 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:10.764451 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:11.586664 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:11.586613 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:14.005182 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:14.005149 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:14.302127 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:14.302040 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:14.476125 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:14.476087 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:15.665197 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:15.665165 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:15.831880 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:15.831830 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:16.024283 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:16.024163 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:16.689445 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:16.689381 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:16.804702 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:16.804653 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:17.896943 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:17.896912 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:45.288516 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:45.288480 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:52.074183 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:52.074151 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:55.708773 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:55.708739 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:56.330738 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:56.330687 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:56.401361 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:56.401324 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:56.499850 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:56.499795 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:56.601159 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:56.601070 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:57.643733 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:57.643701 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:58.049041 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:58.048960 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:58.352736 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:58.352698 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:16:58.862220 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:16:58.862183 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:01.263453 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:01.263412 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:01.361581 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:01.361540 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:02.208578 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:02.208544 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:03.155105 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:03.155064 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:03.278568 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:03.278529 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:03.388062 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:03.387988 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:03.758262 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:03.758230 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:03.858372 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:03.858313 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:04.080226 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.080155 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw"] Sep 10 22:17:04.080459 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.080428 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" podUID="aa96df94-e8c7-4904-99a0-811cccb98aa0" containerName="maas-api" containerID="cri-o://7d72cdefb2d1c0e18fd3382d6b63c56047eaa10da2dad2308d75258b22821971" gracePeriod=30 Sep 10 22:17:04.132779 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.132750 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:04.144484 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.144445 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck"] Sep 10 22:17:04.145174 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.144777 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" podUID="aebe8545-0d97-4c82-843c-7563fb4fb3e8" containerName="payload-processing" containerID="cri-o://49d05203fc041e5b8fbcfc2596dce8849c9ec1f08c32ba05b41cec19127eef35" gracePeriod=30 Sep 10 22:17:04.232785 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.232741 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:04.332885 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.332788 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:04.423396 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.423359 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:17:04.429283 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.429237 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:04.469596 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.469562 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" Sep 10 22:17:04.590879 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.590758 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6c7p\" (UniqueName: \"kubernetes.io/projected/aebe8545-0d97-4c82-843c-7563fb4fb3e8-kube-api-access-r6c7p\") pod \"aebe8545-0d97-4c82-843c-7563fb4fb3e8\" (UID: \"aebe8545-0d97-4c82-843c-7563fb4fb3e8\") " Sep 10 22:17:04.590879 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.590834 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa96df94-e8c7-4904-99a0-811cccb98aa0-metrics-certs\") pod \"aa96df94-e8c7-4904-99a0-811cccb98aa0\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " Sep 10 22:17:04.591108 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.590895 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdh2n\" (UniqueName: \"kubernetes.io/projected/aa96df94-e8c7-4904-99a0-811cccb98aa0-kube-api-access-kdh2n\") pod \"aa96df94-e8c7-4904-99a0-811cccb98aa0\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " Sep 10 22:17:04.591108 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.590926 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa96df94-e8c7-4904-99a0-811cccb98aa0-maas-api-tls\") pod \"aa96df94-e8c7-4904-99a0-811cccb98aa0\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " Sep 10 22:17:04.591108 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.590984 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa96df94-e8c7-4904-99a0-811cccb98aa0-openshift-service-ca\") pod \"aa96df94-e8c7-4904-99a0-811cccb98aa0\" (UID: \"aa96df94-e8c7-4904-99a0-811cccb98aa0\") " Sep 10 22:17:04.591108 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.591025 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aebe8545-0d97-4c82-843c-7563fb4fb3e8-plugins-config-volume\") pod \"aebe8545-0d97-4c82-843c-7563fb4fb3e8\" (UID: \"aebe8545-0d97-4c82-843c-7563fb4fb3e8\") " Sep 10 22:17:04.591649 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.591527 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa96df94-e8c7-4904-99a0-811cccb98aa0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa96df94-e8c7-4904-99a0-811cccb98aa0" (UID: "aa96df94-e8c7-4904-99a0-811cccb98aa0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:17:04.591775 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.591710 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aebe8545-0d97-4c82-843c-7563fb4fb3e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aebe8545-0d97-4c82-843c-7563fb4fb3e8" (UID: "aebe8545-0d97-4c82-843c-7563fb4fb3e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:17:04.593328 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.593299 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa96df94-e8c7-4904-99a0-811cccb98aa0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa96df94-e8c7-4904-99a0-811cccb98aa0" (UID: "aa96df94-e8c7-4904-99a0-811cccb98aa0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:17:04.593328 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.593316 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa96df94-e8c7-4904-99a0-811cccb98aa0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aa96df94-e8c7-4904-99a0-811cccb98aa0" (UID: "aa96df94-e8c7-4904-99a0-811cccb98aa0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:17:04.593477 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.593338 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa96df94-e8c7-4904-99a0-811cccb98aa0-kube-api-access-kdh2n" (OuterVolumeSpecName: "kube-api-access-kdh2n") pod "aa96df94-e8c7-4904-99a0-811cccb98aa0" (UID: "aa96df94-e8c7-4904-99a0-811cccb98aa0"). InnerVolumeSpecName "kube-api-access-kdh2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:17:04.593477 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.593364 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aebe8545-0d97-4c82-843c-7563fb4fb3e8-kube-api-access-r6c7p" (OuterVolumeSpecName: "kube-api-access-r6c7p") pod "aebe8545-0d97-4c82-843c-7563fb4fb3e8" (UID: "aebe8545-0d97-4c82-843c-7563fb4fb3e8"). InnerVolumeSpecName "kube-api-access-r6c7p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:17:04.692548 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.692508 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa96df94-e8c7-4904-99a0-811cccb98aa0-openshift-service-ca\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:17:04.692548 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.692539 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aebe8545-0d97-4c82-843c-7563fb4fb3e8-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:17:04.692548 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.692549 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6c7p\" (UniqueName: \"kubernetes.io/projected/aebe8545-0d97-4c82-843c-7563fb4fb3e8-kube-api-access-r6c7p\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:17:04.692548 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.692561 2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa96df94-e8c7-4904-99a0-811cccb98aa0-metrics-certs\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:17:04.692943 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.692571 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdh2n\" (UniqueName: \"kubernetes.io/projected/aa96df94-e8c7-4904-99a0-811cccb98aa0-kube-api-access-kdh2n\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:17:04.692943 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.692579 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa96df94-e8c7-4904-99a0-811cccb98aa0-maas-api-tls\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:17:04.697270 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.697200 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:04.944326 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.944234 2576 generic.go:356] "Generic (PLEG): container finished" podID="aebe8545-0d97-4c82-843c-7563fb4fb3e8" containerID="49d05203fc041e5b8fbcfc2596dce8849c9ec1f08c32ba05b41cec19127eef35" exitCode=0 Sep 10 22:17:04.944326 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.944297 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" Sep 10 22:17:04.944326 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.944318 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" event={"ID":"aebe8545-0d97-4c82-843c-7563fb4fb3e8","Type":"ContainerDied","Data":"49d05203fc041e5b8fbcfc2596dce8849c9ec1f08c32ba05b41cec19127eef35"} Sep 10 22:17:04.944625 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.944397 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck" event={"ID":"aebe8545-0d97-4c82-843c-7563fb4fb3e8","Type":"ContainerDied","Data":"7cb264d5335bb936efa3316b632f6d34472fb05c8c7cf1a89769de7e9b797a6b"} Sep 10 22:17:04.944625 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.944423 2576 scope.go:117] "RemoveContainer" containerID="49d05203fc041e5b8fbcfc2596dce8849c9ec1f08c32ba05b41cec19127eef35" Sep 10 22:17:04.945833 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.945781 2576 generic.go:356] "Generic (PLEG): container finished" podID="aa96df94-e8c7-4904-99a0-811cccb98aa0" containerID="7d72cdefb2d1c0e18fd3382d6b63c56047eaa10da2dad2308d75258b22821971" exitCode=0 Sep 10 22:17:04.945952 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.945827 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" event={"ID":"aa96df94-e8c7-4904-99a0-811cccb98aa0","Type":"ContainerDied","Data":"7d72cdefb2d1c0e18fd3382d6b63c56047eaa10da2dad2308d75258b22821971"} Sep 10 22:17:04.945952 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.945859 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" Sep 10 22:17:04.945952 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.945878 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw" event={"ID":"aa96df94-e8c7-4904-99a0-811cccb98aa0","Type":"ContainerDied","Data":"90d5a2a296187451d4d4231f40367cab1b85c71a095fc1461ee55b02c157ad4f"} Sep 10 22:17:04.954863 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.954844 2576 scope.go:117] "RemoveContainer" containerID="49d05203fc041e5b8fbcfc2596dce8849c9ec1f08c32ba05b41cec19127eef35" Sep 10 22:17:04.955234 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:17:04.955212 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49d05203fc041e5b8fbcfc2596dce8849c9ec1f08c32ba05b41cec19127eef35\": container with ID starting with 49d05203fc041e5b8fbcfc2596dce8849c9ec1f08c32ba05b41cec19127eef35 not found: ID does not exist" containerID="49d05203fc041e5b8fbcfc2596dce8849c9ec1f08c32ba05b41cec19127eef35" Sep 10 22:17:04.955296 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.955244 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49d05203fc041e5b8fbcfc2596dce8849c9ec1f08c32ba05b41cec19127eef35"} err="failed to get container status \"49d05203fc041e5b8fbcfc2596dce8849c9ec1f08c32ba05b41cec19127eef35\": rpc error: code = NotFound desc = could not find container \"49d05203fc041e5b8fbcfc2596dce8849c9ec1f08c32ba05b41cec19127eef35\": container with ID starting with 49d05203fc041e5b8fbcfc2596dce8849c9ec1f08c32ba05b41cec19127eef35 not found: ID does not exist" Sep 10 22:17:04.955296 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.955264 2576 scope.go:117] "RemoveContainer" containerID="7d72cdefb2d1c0e18fd3382d6b63c56047eaa10da2dad2308d75258b22821971" Sep 10 22:17:04.964868 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.964841 2576 scope.go:117] "RemoveContainer" containerID="7d72cdefb2d1c0e18fd3382d6b63c56047eaa10da2dad2308d75258b22821971" Sep 10 22:17:04.965333 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:17:04.965311 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d72cdefb2d1c0e18fd3382d6b63c56047eaa10da2dad2308d75258b22821971\": container with ID starting with 7d72cdefb2d1c0e18fd3382d6b63c56047eaa10da2dad2308d75258b22821971 not found: ID does not exist" containerID="7d72cdefb2d1c0e18fd3382d6b63c56047eaa10da2dad2308d75258b22821971" Sep 10 22:17:04.965434 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.965344 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d72cdefb2d1c0e18fd3382d6b63c56047eaa10da2dad2308d75258b22821971"} err="failed to get container status \"7d72cdefb2d1c0e18fd3382d6b63c56047eaa10da2dad2308d75258b22821971\": rpc error: code = NotFound desc = could not find container \"7d72cdefb2d1c0e18fd3382d6b63c56047eaa10da2dad2308d75258b22821971\": container with ID starting with 7d72cdefb2d1c0e18fd3382d6b63c56047eaa10da2dad2308d75258b22821971 not found: ID does not exist" Sep 10 22:17:04.966785 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.966761 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck"] Sep 10 22:17:04.972664 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.972634 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-d6a4de-77c99bcd7c-f78ck"] Sep 10 22:17:04.982091 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.982057 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw"] Sep 10 22:17:04.994500 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:04.994469 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-d6a4de-5978677b7-4p2hw"] Sep 10 22:17:06.432465 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:06.432434 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa96df94-e8c7-4904-99a0-811cccb98aa0" path="/var/lib/kubelet/pods/aa96df94-e8c7-4904-99a0-811cccb98aa0/volumes" Sep 10 22:17:06.432883 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:06.432837 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aebe8545-0d97-4c82-843c-7563fb4fb3e8" path="/var/lib/kubelet/pods/aebe8545-0d97-4c82-843c-7563fb4fb3e8/volumes" Sep 10 22:17:09.773296 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:09.773253 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:12.879173 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:12.879141 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:14.100701 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:14.100665 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:20.202399 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:20.202364 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:24.294645 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:24.294591 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:33.067021 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:33.066932 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:33.145867 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:33.145832 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:33.215974 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:33.215936 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:33.286917 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:33.286876 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:33.353782 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:33.353699 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:33.430104 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:33.430046 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:34.081966 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:34.081911 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:34.177298 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:34.177265 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:34.280844 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:34.280789 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:35.588520 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:35.588485 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:42.237076 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:42.237031 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:42.347664 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:42.347630 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:42.442006 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:42.441968 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:42.542973 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:42.542889 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:42.929532 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:42.929491 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:46.869034 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:46.868973 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:47.132243 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:47.132131 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:47.553820 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:47.553702 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:47.866326 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:47.866272 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:47.973265 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:47.973216 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:50.894839 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:50.894783 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:51.143613 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:51.143576 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:51.238937 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:51.238784 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:51.704576 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:51.704538 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:52.377845 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:52.377786 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:52.450036 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:52.449954 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:52.871730 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:52.871702 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:17:52.941856 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:17:52.941789 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:03.051071 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:03.051040 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:08.833014 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:08.832982 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:14.888204 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:14.888174 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:14.995996 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:14.995961 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:15.089454 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:15.089421 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:16.060603 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:16.060567 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:17.918209 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:17.918150 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:31.017850 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:31.017795 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:37.106198 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:37.106159 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:37.170574 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:37.170543 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:37.258375 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:37.258341 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:37.358603 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:37.358518 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:37.797030 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:37.796942 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:45.901141 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:45.901085 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:47.799997 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:47.799966 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:48.530086 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:48.530046 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:50.280924 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:50.280893 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:54.600635 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:54.600600 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:54.695238 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:54.695208 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:54.795601 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:54.795547 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:54.911877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:54.911772 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:55.508828 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:55.508779 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:55.608069 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:55.608036 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:55.708324 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:55.708267 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:56.562955 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:56.562906 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:56.663757 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:56.663720 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:56.763545 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:56.763510 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:56.862327 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:56.862280 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:57.125601 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:57.125504 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:18:57.226996 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:18:57.226962 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:03.737966 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:03.737882 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:03.835569 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:03.835520 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:03.961348 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:03.961315 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:04.062121 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:04.062030 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:04.441607 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:04.441574 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:04.637272 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:04.637239 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:04.715529 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:04.715446 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:04.873656 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:04.873619 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:06.580555 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:06.580517 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:06.679257 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:06.679223 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:06.780187 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:06.780159 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:08.247250 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:08.247219 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:08.302122 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:08.302093 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:10.448231 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:10.448193 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:10.548956 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:10.548926 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:10.667346 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:10.667315 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:11.000315 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:11.000281 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:11.291623 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:11.291539 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:13.711374 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:13.711347 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:13.850295 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:13.850262 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:14.901495 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:14.901454 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:15.009131 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.009082 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:15.248400 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.248303 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:15.423122 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.423091 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:15.522559 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.522456 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:15.565077 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.565041 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5"] Sep 10 22:19:15.565565 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.565548 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aebe8545-0d97-4c82-843c-7563fb4fb3e8" containerName="payload-processing" Sep 10 22:19:15.565622 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.565567 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aebe8545-0d97-4c82-843c-7563fb4fb3e8" containerName="payload-processing" Sep 10 22:19:15.565622 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.565583 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa96df94-e8c7-4904-99a0-811cccb98aa0" containerName="maas-api" Sep 10 22:19:15.565622 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.565588 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa96df94-e8c7-4904-99a0-811cccb98aa0" containerName="maas-api" Sep 10 22:19:15.565730 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.565654 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aebe8545-0d97-4c82-843c-7563fb4fb3e8" containerName="payload-processing" Sep 10 22:19:15.565730 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.565663 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa96df94-e8c7-4904-99a0-811cccb98aa0" containerName="maas-api" Sep 10 22:19:15.568577 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.568562 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" Sep 10 22:19:15.571121 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.571100 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-d98bb58d-dockercfg-l6lkv\"" Sep 10 22:19:15.571351 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.571338 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-d98bb58d\"" Sep 10 22:19:15.579948 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.579917 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5"] Sep 10 22:19:15.598407 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.598355 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk7zv\" (UniqueName: \"kubernetes.io/projected/56c5bb6c-82e6-4eb6-bf62-036fc68ec72e-kube-api-access-rk7zv\") pod \"payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5\" (UID: \"56c5bb6c-82e6-4eb6-bf62-036fc68ec72e\") " pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" Sep 10 22:19:15.598891 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.598559 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56c5bb6c-82e6-4eb6-bf62-036fc68ec72e-plugins-config-volume\") pod \"payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5\" (UID: \"56c5bb6c-82e6-4eb6-bf62-036fc68ec72e\") " pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" Sep 10 22:19:15.699244 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.699200 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56c5bb6c-82e6-4eb6-bf62-036fc68ec72e-plugins-config-volume\") pod \"payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5\" (UID: \"56c5bb6c-82e6-4eb6-bf62-036fc68ec72e\") " pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" Sep 10 22:19:15.699424 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.699315 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk7zv\" (UniqueName: \"kubernetes.io/projected/56c5bb6c-82e6-4eb6-bf62-036fc68ec72e-kube-api-access-rk7zv\") pod \"payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5\" (UID: \"56c5bb6c-82e6-4eb6-bf62-036fc68ec72e\") " pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" Sep 10 22:19:15.700138 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.700113 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56c5bb6c-82e6-4eb6-bf62-036fc68ec72e-plugins-config-volume\") pod \"payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5\" (UID: \"56c5bb6c-82e6-4eb6-bf62-036fc68ec72e\") " pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" Sep 10 22:19:15.709932 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.709899 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk7zv\" (UniqueName: \"kubernetes.io/projected/56c5bb6c-82e6-4eb6-bf62-036fc68ec72e-kube-api-access-rk7zv\") pod \"payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5\" (UID: \"56c5bb6c-82e6-4eb6-bf62-036fc68ec72e\") " pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" Sep 10 22:19:15.880362 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:15.880320 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" Sep 10 22:19:16.158328 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:16.158280 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5"] Sep 10 22:19:16.160537 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:19:16.160511 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56c5bb6c_82e6_4eb6_bf62_036fc68ec72e.slice/crio-92422fef23c05ef4c41d06b1209eba14dbb2daba14d4708a4870d3e55689d225 WatchSource:0}: Error finding container 92422fef23c05ef4c41d06b1209eba14dbb2daba14d4708a4870d3e55689d225: Status 404 returned error can't find the container with id 92422fef23c05ef4c41d06b1209eba14dbb2daba14d4708a4870d3e55689d225 Sep 10 22:19:16.644510 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:16.644480 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" event={"ID":"56c5bb6c-82e6-4eb6-bf62-036fc68ec72e","Type":"ContainerStarted","Data":"e91ab229eb49c1ef43f8697661d6a8fff5e40a259b8e99501c4778bd2a75718c"} Sep 10 22:19:16.644704 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:16.644529 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" event={"ID":"56c5bb6c-82e6-4eb6-bf62-036fc68ec72e","Type":"ContainerStarted","Data":"92422fef23c05ef4c41d06b1209eba14dbb2daba14d4708a4870d3e55689d225"} Sep 10 22:19:16.644704 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:16.644588 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" Sep 10 22:19:16.661154 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:16.661092 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" podStartSLOduration=1.661070279 podStartE2EDuration="1.661070279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:16.659553248 +0000 UTC m=+1086.781252032" watchObservedRunningTime="2026-09-10 22:19:16.661070279 +0000 UTC m=+1086.782769061" Sep 10 22:19:27.651303 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:27.651262 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" Sep 10 22:19:38.661334 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:38.661296 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:39.292651 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:39.292606 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:39.384387 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:39.384359 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:39.690448 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:39.690415 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:41.125864 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:41.125834 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:44.157339 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:44.157306 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:45.000491 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:45.000459 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:45.123108 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:45.123068 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:45.343970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:45.343932 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:45.444028 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:45.443996 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:46.002639 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:46.002601 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:46.148695 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:46.148664 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:46.246601 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:46.246563 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:46.404727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:46.404688 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:47.198195 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:47.198165 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:48.096355 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:48.096300 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:50.570857 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:50.570791 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:56.658527 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:56.658492 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:56.685061 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:56.685018 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5"] Sep 10 22:19:56.685324 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:56.685295 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" podUID="56c5bb6c-82e6-4eb6-bf62-036fc68ec72e" containerName="payload-processing" containerID="cri-o://e91ab229eb49c1ef43f8697661d6a8fff5e40a259b8e99501c4778bd2a75718c" gracePeriod=30 Sep 10 22:19:56.762763 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:56.762728 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:56.798665 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:56.798630 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7"] Sep 10 22:19:56.798892 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:56.798866 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" podUID="72feb2e1-36b7-4a0f-b883-5f238cedecf5" containerName="maas-api" containerID="cri-o://ece66bcbef81e39825ee20c880b7c3e76aca39f883a040edaf4542ea5b76e171" gracePeriod=30 Sep 10 22:19:56.858087 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:56.858053 2576 generic.go:356] "Generic (PLEG): container finished" podID="56c5bb6c-82e6-4eb6-bf62-036fc68ec72e" containerID="e91ab229eb49c1ef43f8697661d6a8fff5e40a259b8e99501c4778bd2a75718c" exitCode=0 Sep 10 22:19:56.858264 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:56.858110 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" event={"ID":"56c5bb6c-82e6-4eb6-bf62-036fc68ec72e","Type":"ContainerDied","Data":"e91ab229eb49c1ef43f8697661d6a8fff5e40a259b8e99501c4778bd2a75718c"} Sep 10 22:19:56.861860 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:56.861828 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:56.887878 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:56.887812 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w"] Sep 10 22:19:56.888610 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:56.888567 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" podUID="20953f3c-7e42-4683-8f19-ef7379cd2c04" containerName="payload-pre-processing" containerID="cri-o://dee558c9363faf7800fd4e46444d6dc74cc2623cc7aeebf0a489cf3b95d912a7" gracePeriod=30 Sep 10 22:19:56.894780 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:56.894587 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf"] Sep 10 22:19:56.894949 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:56.894837 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" podUID="cb47e7b6-331c-41c5-8b24-3c1533b7fd09" containerName="payload-processing" containerID="cri-o://9c1ec94a143bd0f17357b64cab252fe94b759c82b629f42e51e47dc257ba031b" gracePeriod=30 Sep 10 22:19:56.939024 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:56.938984 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:57.030142 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.029851 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" Sep 10 22:19:57.062910 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.062844 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:57.125292 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.125253 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:57.130465 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.130436 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56c5bb6c-82e6-4eb6-bf62-036fc68ec72e-plugins-config-volume\") pod \"56c5bb6c-82e6-4eb6-bf62-036fc68ec72e\" (UID: \"56c5bb6c-82e6-4eb6-bf62-036fc68ec72e\") " Sep 10 22:19:57.130641 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.130487 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rk7zv\" (UniqueName: \"kubernetes.io/projected/56c5bb6c-82e6-4eb6-bf62-036fc68ec72e-kube-api-access-rk7zv\") pod \"56c5bb6c-82e6-4eb6-bf62-036fc68ec72e\" (UID: \"56c5bb6c-82e6-4eb6-bf62-036fc68ec72e\") " Sep 10 22:19:57.130908 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.130887 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56c5bb6c-82e6-4eb6-bf62-036fc68ec72e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "56c5bb6c-82e6-4eb6-bf62-036fc68ec72e" (UID: "56c5bb6c-82e6-4eb6-bf62-036fc68ec72e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:19:57.134078 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.134044 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56c5bb6c-82e6-4eb6-bf62-036fc68ec72e-kube-api-access-rk7zv" (OuterVolumeSpecName: "kube-api-access-rk7zv") pod "56c5bb6c-82e6-4eb6-bf62-036fc68ec72e" (UID: "56c5bb6c-82e6-4eb6-bf62-036fc68ec72e"). InnerVolumeSpecName "kube-api-access-rk7zv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:19:57.178581 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.178545 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:19:57.182262 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.182233 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" Sep 10 22:19:57.208213 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.208181 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" Sep 10 22:19:57.231428 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.231397 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72feb2e1-36b7-4a0f-b883-5f238cedecf5-maas-api-tls\") pod \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " Sep 10 22:19:57.231609 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.231439 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72feb2e1-36b7-4a0f-b883-5f238cedecf5-openshift-service-ca\") pod \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " Sep 10 22:19:57.231609 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.231580 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnhbh\" (UniqueName: \"kubernetes.io/projected/72feb2e1-36b7-4a0f-b883-5f238cedecf5-kube-api-access-jnhbh\") pod \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " Sep 10 22:19:57.231733 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.231643 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72feb2e1-36b7-4a0f-b883-5f238cedecf5-metrics-certs\") pod \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\" (UID: \"72feb2e1-36b7-4a0f-b883-5f238cedecf5\") " Sep 10 22:19:57.231733 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.231675 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89spv\" (UniqueName: \"kubernetes.io/projected/cb47e7b6-331c-41c5-8b24-3c1533b7fd09-kube-api-access-89spv\") pod \"cb47e7b6-331c-41c5-8b24-3c1533b7fd09\" (UID: \"cb47e7b6-331c-41c5-8b24-3c1533b7fd09\") " Sep 10 22:19:57.231733 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.231712 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb47e7b6-331c-41c5-8b24-3c1533b7fd09-plugins-config-volume\") pod \"cb47e7b6-331c-41c5-8b24-3c1533b7fd09\" (UID: \"cb47e7b6-331c-41c5-8b24-3c1533b7fd09\") " Sep 10 22:19:57.231898 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.231831 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72feb2e1-36b7-4a0f-b883-5f238cedecf5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "72feb2e1-36b7-4a0f-b883-5f238cedecf5" (UID: "72feb2e1-36b7-4a0f-b883-5f238cedecf5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:19:57.232087 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.232068 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56c5bb6c-82e6-4eb6-bf62-036fc68ec72e-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:19:57.232158 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.232091 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rk7zv\" (UniqueName: \"kubernetes.io/projected/56c5bb6c-82e6-4eb6-bf62-036fc68ec72e-kube-api-access-rk7zv\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:19:57.232158 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.232105 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72feb2e1-36b7-4a0f-b883-5f238cedecf5-openshift-service-ca\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:19:57.232273 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.232251 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb47e7b6-331c-41c5-8b24-3c1533b7fd09-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cb47e7b6-331c-41c5-8b24-3c1533b7fd09" (UID: "cb47e7b6-331c-41c5-8b24-3c1533b7fd09"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:19:57.234238 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.234044 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb47e7b6-331c-41c5-8b24-3c1533b7fd09-kube-api-access-89spv" (OuterVolumeSpecName: "kube-api-access-89spv") pod "cb47e7b6-331c-41c5-8b24-3c1533b7fd09" (UID: "cb47e7b6-331c-41c5-8b24-3c1533b7fd09"). InnerVolumeSpecName "kube-api-access-89spv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:19:57.234238 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.234125 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72feb2e1-36b7-4a0f-b883-5f238cedecf5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "72feb2e1-36b7-4a0f-b883-5f238cedecf5" (UID: "72feb2e1-36b7-4a0f-b883-5f238cedecf5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:19:57.234238 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.234148 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72feb2e1-36b7-4a0f-b883-5f238cedecf5-kube-api-access-jnhbh" (OuterVolumeSpecName: "kube-api-access-jnhbh") pod "72feb2e1-36b7-4a0f-b883-5f238cedecf5" (UID: "72feb2e1-36b7-4a0f-b883-5f238cedecf5"). InnerVolumeSpecName "kube-api-access-jnhbh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:19:57.234406 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.234241 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72feb2e1-36b7-4a0f-b883-5f238cedecf5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "72feb2e1-36b7-4a0f-b883-5f238cedecf5" (UID: "72feb2e1-36b7-4a0f-b883-5f238cedecf5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:19:57.291217 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.291179 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:57.332444 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.332407 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd24h\" (UniqueName: \"kubernetes.io/projected/20953f3c-7e42-4683-8f19-ef7379cd2c04-kube-api-access-gd24h\") pod \"20953f3c-7e42-4683-8f19-ef7379cd2c04\" (UID: \"20953f3c-7e42-4683-8f19-ef7379cd2c04\") " Sep 10 22:19:57.332619 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.332518 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20953f3c-7e42-4683-8f19-ef7379cd2c04-plugins-config-volume\") pod \"20953f3c-7e42-4683-8f19-ef7379cd2c04\" (UID: \"20953f3c-7e42-4683-8f19-ef7379cd2c04\") " Sep 10 22:19:57.332694 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.332682 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72feb2e1-36b7-4a0f-b883-5f238cedecf5-maas-api-tls\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:19:57.332730 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.332697 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnhbh\" (UniqueName: \"kubernetes.io/projected/72feb2e1-36b7-4a0f-b883-5f238cedecf5-kube-api-access-jnhbh\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:19:57.332730 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.332706 2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72feb2e1-36b7-4a0f-b883-5f238cedecf5-metrics-certs\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:19:57.332730 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.332715 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89spv\" (UniqueName: \"kubernetes.io/projected/cb47e7b6-331c-41c5-8b24-3c1533b7fd09-kube-api-access-89spv\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:19:57.332730 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.332725 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb47e7b6-331c-41c5-8b24-3c1533b7fd09-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:19:57.333011 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.332917 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20953f3c-7e42-4683-8f19-ef7379cd2c04-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "20953f3c-7e42-4683-8f19-ef7379cd2c04" (UID: "20953f3c-7e42-4683-8f19-ef7379cd2c04"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:19:57.334585 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.334562 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20953f3c-7e42-4683-8f19-ef7379cd2c04-kube-api-access-gd24h" (OuterVolumeSpecName: "kube-api-access-gd24h") pod "20953f3c-7e42-4683-8f19-ef7379cd2c04" (UID: "20953f3c-7e42-4683-8f19-ef7379cd2c04"). InnerVolumeSpecName "kube-api-access-gd24h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:19:57.433742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.433654 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20953f3c-7e42-4683-8f19-ef7379cd2c04-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:19:57.433742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.433683 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd24h\" (UniqueName: \"kubernetes.io/projected/20953f3c-7e42-4683-8f19-ef7379cd2c04-kube-api-access-gd24h\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:19:57.468487 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.468455 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:19:57.864653 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.864612 2576 generic.go:356] "Generic (PLEG): container finished" podID="20953f3c-7e42-4683-8f19-ef7379cd2c04" containerID="dee558c9363faf7800fd4e46444d6dc74cc2623cc7aeebf0a489cf3b95d912a7" exitCode=0 Sep 10 22:19:57.865159 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.864660 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" Sep 10 22:19:57.865159 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.864699 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" event={"ID":"20953f3c-7e42-4683-8f19-ef7379cd2c04","Type":"ContainerDied","Data":"dee558c9363faf7800fd4e46444d6dc74cc2623cc7aeebf0a489cf3b95d912a7"} Sep 10 22:19:57.865159 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.864753 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w" event={"ID":"20953f3c-7e42-4683-8f19-ef7379cd2c04","Type":"ContainerDied","Data":"d17df422a6b4f6a8467050029c0c76259054edc725815434d9989f835fc4935e"} Sep 10 22:19:57.865159 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.864785 2576 scope.go:117] "RemoveContainer" containerID="dee558c9363faf7800fd4e46444d6dc74cc2623cc7aeebf0a489cf3b95d912a7" Sep 10 22:19:57.866157 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.866054 2576 generic.go:356] "Generic (PLEG): container finished" podID="72feb2e1-36b7-4a0f-b883-5f238cedecf5" containerID="ece66bcbef81e39825ee20c880b7c3e76aca39f883a040edaf4542ea5b76e171" exitCode=0 Sep 10 22:19:57.866157 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.866086 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" event={"ID":"72feb2e1-36b7-4a0f-b883-5f238cedecf5","Type":"ContainerDied","Data":"ece66bcbef81e39825ee20c880b7c3e76aca39f883a040edaf4542ea5b76e171"} Sep 10 22:19:57.866157 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.866106 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" Sep 10 22:19:57.866157 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.866135 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7" event={"ID":"72feb2e1-36b7-4a0f-b883-5f238cedecf5","Type":"ContainerDied","Data":"0abc07dfcfb0dfeca27ccb604f6e12a013b1764d986d111a2dabf686fba0c673"} Sep 10 22:19:57.867771 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.867752 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" Sep 10 22:19:57.867921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.867789 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5" event={"ID":"56c5bb6c-82e6-4eb6-bf62-036fc68ec72e","Type":"ContainerDied","Data":"92422fef23c05ef4c41d06b1209eba14dbb2daba14d4708a4870d3e55689d225"} Sep 10 22:19:57.869438 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.869401 2576 generic.go:356] "Generic (PLEG): container finished" podID="cb47e7b6-331c-41c5-8b24-3c1533b7fd09" containerID="9c1ec94a143bd0f17357b64cab252fe94b759c82b629f42e51e47dc257ba031b" exitCode=0 Sep 10 22:19:57.869555 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.869445 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" Sep 10 22:19:57.869555 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.869535 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" event={"ID":"cb47e7b6-331c-41c5-8b24-3c1533b7fd09","Type":"ContainerDied","Data":"9c1ec94a143bd0f17357b64cab252fe94b759c82b629f42e51e47dc257ba031b"} Sep 10 22:19:57.869672 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.869581 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf" event={"ID":"cb47e7b6-331c-41c5-8b24-3c1533b7fd09","Type":"ContainerDied","Data":"701253d63f19b978e7f962003714a58a3907932e20a65bcf66b46b212f8b27c7"} Sep 10 22:19:57.875885 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.875855 2576 scope.go:117] "RemoveContainer" containerID="dee558c9363faf7800fd4e46444d6dc74cc2623cc7aeebf0a489cf3b95d912a7" Sep 10 22:19:57.876242 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:19:57.876222 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dee558c9363faf7800fd4e46444d6dc74cc2623cc7aeebf0a489cf3b95d912a7\": container with ID starting with dee558c9363faf7800fd4e46444d6dc74cc2623cc7aeebf0a489cf3b95d912a7 not found: ID does not exist" containerID="dee558c9363faf7800fd4e46444d6dc74cc2623cc7aeebf0a489cf3b95d912a7" Sep 10 22:19:57.876328 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.876249 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dee558c9363faf7800fd4e46444d6dc74cc2623cc7aeebf0a489cf3b95d912a7"} err="failed to get container status \"dee558c9363faf7800fd4e46444d6dc74cc2623cc7aeebf0a489cf3b95d912a7\": rpc error: code = NotFound desc = could not find container \"dee558c9363faf7800fd4e46444d6dc74cc2623cc7aeebf0a489cf3b95d912a7\": container with ID starting with dee558c9363faf7800fd4e46444d6dc74cc2623cc7aeebf0a489cf3b95d912a7 not found: ID does not exist" Sep 10 22:19:57.876328 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.876266 2576 scope.go:117] "RemoveContainer" containerID="ece66bcbef81e39825ee20c880b7c3e76aca39f883a040edaf4542ea5b76e171" Sep 10 22:19:57.886405 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.886384 2576 scope.go:117] "RemoveContainer" containerID="ece66bcbef81e39825ee20c880b7c3e76aca39f883a040edaf4542ea5b76e171" Sep 10 22:19:57.886735 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:19:57.886712 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ece66bcbef81e39825ee20c880b7c3e76aca39f883a040edaf4542ea5b76e171\": container with ID starting with ece66bcbef81e39825ee20c880b7c3e76aca39f883a040edaf4542ea5b76e171 not found: ID does not exist" containerID="ece66bcbef81e39825ee20c880b7c3e76aca39f883a040edaf4542ea5b76e171" Sep 10 22:19:57.886859 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.886740 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ece66bcbef81e39825ee20c880b7c3e76aca39f883a040edaf4542ea5b76e171"} err="failed to get container status \"ece66bcbef81e39825ee20c880b7c3e76aca39f883a040edaf4542ea5b76e171\": rpc error: code = NotFound desc = could not find container \"ece66bcbef81e39825ee20c880b7c3e76aca39f883a040edaf4542ea5b76e171\": container with ID starting with ece66bcbef81e39825ee20c880b7c3e76aca39f883a040edaf4542ea5b76e171 not found: ID does not exist" Sep 10 22:19:57.886859 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.886758 2576 scope.go:117] "RemoveContainer" containerID="e91ab229eb49c1ef43f8697661d6a8fff5e40a259b8e99501c4778bd2a75718c" Sep 10 22:19:57.893089 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.893062 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7"] Sep 10 22:19:57.899175 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.899142 2576 scope.go:117] "RemoveContainer" containerID="9c1ec94a143bd0f17357b64cab252fe94b759c82b629f42e51e47dc257ba031b" Sep 10 22:19:57.900238 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.900210 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e5a2d7-6d5db87f65-qltz7"] Sep 10 22:19:57.908935 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.908502 2576 scope.go:117] "RemoveContainer" containerID="9c1ec94a143bd0f17357b64cab252fe94b759c82b629f42e51e47dc257ba031b" Sep 10 22:19:57.909104 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:19:57.909079 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c1ec94a143bd0f17357b64cab252fe94b759c82b629f42e51e47dc257ba031b\": container with ID starting with 9c1ec94a143bd0f17357b64cab252fe94b759c82b629f42e51e47dc257ba031b not found: ID does not exist" containerID="9c1ec94a143bd0f17357b64cab252fe94b759c82b629f42e51e47dc257ba031b" Sep 10 22:19:57.909194 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.909116 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c1ec94a143bd0f17357b64cab252fe94b759c82b629f42e51e47dc257ba031b"} err="failed to get container status \"9c1ec94a143bd0f17357b64cab252fe94b759c82b629f42e51e47dc257ba031b\": rpc error: code = NotFound desc = could not find container \"9c1ec94a143bd0f17357b64cab252fe94b759c82b629f42e51e47dc257ba031b\": container with ID starting with 9c1ec94a143bd0f17357b64cab252fe94b759c82b629f42e51e47dc257ba031b not found: ID does not exist" Sep 10 22:19:57.910879 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.910855 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf"] Sep 10 22:19:57.917999 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.917973 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e5a2d7-56dddc7445-pqrkf"] Sep 10 22:19:57.929887 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.929860 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w"] Sep 10 22:19:57.935441 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.935409 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e5a2d7-6bdbf75c54-8ms6w"] Sep 10 22:19:57.946860 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.946829 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5"] Sep 10 22:19:57.949621 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:57.949595 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d98bb58d-6694ffc689-852q5"] Sep 10 22:19:58.432764 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:58.432732 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20953f3c-7e42-4683-8f19-ef7379cd2c04" path="/var/lib/kubelet/pods/20953f3c-7e42-4683-8f19-ef7379cd2c04/volumes" Sep 10 22:19:58.433147 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:58.433133 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56c5bb6c-82e6-4eb6-bf62-036fc68ec72e" path="/var/lib/kubelet/pods/56c5bb6c-82e6-4eb6-bf62-036fc68ec72e/volumes" Sep 10 22:19:58.433491 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:58.433478 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72feb2e1-36b7-4a0f-b883-5f238cedecf5" path="/var/lib/kubelet/pods/72feb2e1-36b7-4a0f-b883-5f238cedecf5/volumes" Sep 10 22:19:58.433855 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:19:58.433836 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb47e7b6-331c-41c5-8b24-3c1533b7fd09" path="/var/lib/kubelet/pods/cb47e7b6-331c-41c5-8b24-3c1533b7fd09/volumes" Sep 10 22:20:03.545572 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:03.545533 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:03.645511 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:03.645459 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:03.762665 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:03.762632 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:03.859829 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:03.859787 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:03.998607 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:03.998574 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:04.156576 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:04.156455 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:04.256390 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:04.256358 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:04.483116 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:04.483019 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:04.622400 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:04.622372 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:10.641220 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:10.641186 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:10.728492 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:10.728397 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:10.841535 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:10.841493 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:10.931074 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:10.931042 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:11.030020 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:11.029926 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:15.239832 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.239768 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:15.289166 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.289120 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8"] Sep 10 22:20:15.289427 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.289395 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" podUID="fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8" containerName="payload-processing" containerID="cri-o://d7f4926511e27d1b88cdd567f1a535f340957b37556cdf444f10989c362771da" gracePeriod=30 Sep 10 22:20:15.299900 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.299861 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl"] Sep 10 22:20:15.300185 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.300108 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" podUID="e6dc39f0-3ceb-4527-9b17-22cc8526d6c9" containerName="payload-pre-processing" containerID="cri-o://afb2c2d5da65cf1add80560664b59d8ea64e6647a3c7c464a48cef8a0ccb1565" gracePeriod=30 Sep 10 22:20:15.367285 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.367254 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:15.459533 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.459494 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:15.564323 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.564298 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" Sep 10 22:20:15.569687 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.569662 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:15.591710 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.591676 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" Sep 10 22:20:15.616896 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.616861 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbwhv\" (UniqueName: \"kubernetes.io/projected/fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8-kube-api-access-xbwhv\") pod \"fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8\" (UID: \"fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8\") " Sep 10 22:20:15.617482 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.616909 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gvlr\" (UniqueName: \"kubernetes.io/projected/e6dc39f0-3ceb-4527-9b17-22cc8526d6c9-kube-api-access-5gvlr\") pod \"e6dc39f0-3ceb-4527-9b17-22cc8526d6c9\" (UID: \"e6dc39f0-3ceb-4527-9b17-22cc8526d6c9\") " Sep 10 22:20:15.617482 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.616976 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6dc39f0-3ceb-4527-9b17-22cc8526d6c9-plugins-config-volume\") pod \"e6dc39f0-3ceb-4527-9b17-22cc8526d6c9\" (UID: \"e6dc39f0-3ceb-4527-9b17-22cc8526d6c9\") " Sep 10 22:20:15.619039 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.618634 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8-plugins-config-volume\") pod \"fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8\" (UID: \"fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8\") " Sep 10 22:20:15.619039 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.619004 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8" (UID: "fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:20:15.619733 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.619704 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6dc39f0-3ceb-4527-9b17-22cc8526d6c9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6dc39f0-3ceb-4527-9b17-22cc8526d6c9" (UID: "e6dc39f0-3ceb-4527-9b17-22cc8526d6c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:20:15.620207 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.620171 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8-kube-api-access-xbwhv" (OuterVolumeSpecName: "kube-api-access-xbwhv") pod "fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8" (UID: "fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8"). InnerVolumeSpecName "kube-api-access-xbwhv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:20:15.620329 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.620220 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6dc39f0-3ceb-4527-9b17-22cc8526d6c9-kube-api-access-5gvlr" (OuterVolumeSpecName: "kube-api-access-5gvlr") pod "e6dc39f0-3ceb-4527-9b17-22cc8526d6c9" (UID: "e6dc39f0-3ceb-4527-9b17-22cc8526d6c9"). InnerVolumeSpecName "kube-api-access-5gvlr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:20:15.662584 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.662553 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:15.719880 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.719840 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6dc39f0-3ceb-4527-9b17-22cc8526d6c9-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:20:15.719880 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.719873 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:20:15.719880 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.719884 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbwhv\" (UniqueName: \"kubernetes.io/projected/fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8-kube-api-access-xbwhv\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:20:15.719880 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.719892 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gvlr\" (UniqueName: \"kubernetes.io/projected/e6dc39f0-3ceb-4527-9b17-22cc8526d6c9-kube-api-access-5gvlr\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:20:15.930692 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.930646 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:15.966325 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.966294 2576 generic.go:356] "Generic (PLEG): container finished" podID="fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8" containerID="d7f4926511e27d1b88cdd567f1a535f340957b37556cdf444f10989c362771da" exitCode=0 Sep 10 22:20:15.966514 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.966342 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" Sep 10 22:20:15.966514 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.966376 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" event={"ID":"fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8","Type":"ContainerDied","Data":"d7f4926511e27d1b88cdd567f1a535f340957b37556cdf444f10989c362771da"} Sep 10 22:20:15.966514 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.966427 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8" event={"ID":"fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8","Type":"ContainerDied","Data":"5b7c2cbf4f8cb5b352ddcedd3834e35875554a6ecc0a45f892f75b94dde1bcb5"} Sep 10 22:20:15.966514 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.966458 2576 scope.go:117] "RemoveContainer" containerID="d7f4926511e27d1b88cdd567f1a535f340957b37556cdf444f10989c362771da" Sep 10 22:20:15.967942 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.967921 2576 generic.go:356] "Generic (PLEG): container finished" podID="e6dc39f0-3ceb-4527-9b17-22cc8526d6c9" containerID="afb2c2d5da65cf1add80560664b59d8ea64e6647a3c7c464a48cef8a0ccb1565" exitCode=0 Sep 10 22:20:15.968057 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.967962 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" event={"ID":"e6dc39f0-3ceb-4527-9b17-22cc8526d6c9","Type":"ContainerDied","Data":"afb2c2d5da65cf1add80560664b59d8ea64e6647a3c7c464a48cef8a0ccb1565"} Sep 10 22:20:15.968057 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.967969 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" Sep 10 22:20:15.968057 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.967999 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl" event={"ID":"e6dc39f0-3ceb-4527-9b17-22cc8526d6c9","Type":"ContainerDied","Data":"b3a9ea62e55d2838720a703a2b87046e50879418d5e69f2e7a05ae0ff60f7ff7"} Sep 10 22:20:15.976662 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.976643 2576 scope.go:117] "RemoveContainer" containerID="d7f4926511e27d1b88cdd567f1a535f340957b37556cdf444f10989c362771da" Sep 10 22:20:15.977047 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:20:15.977022 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7f4926511e27d1b88cdd567f1a535f340957b37556cdf444f10989c362771da\": container with ID starting with d7f4926511e27d1b88cdd567f1a535f340957b37556cdf444f10989c362771da not found: ID does not exist" containerID="d7f4926511e27d1b88cdd567f1a535f340957b37556cdf444f10989c362771da" Sep 10 22:20:15.977138 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.977056 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7f4926511e27d1b88cdd567f1a535f340957b37556cdf444f10989c362771da"} err="failed to get container status \"d7f4926511e27d1b88cdd567f1a535f340957b37556cdf444f10989c362771da\": rpc error: code = NotFound desc = could not find container \"d7f4926511e27d1b88cdd567f1a535f340957b37556cdf444f10989c362771da\": container with ID starting with d7f4926511e27d1b88cdd567f1a535f340957b37556cdf444f10989c362771da not found: ID does not exist" Sep 10 22:20:15.977138 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.977076 2576 scope.go:117] "RemoveContainer" containerID="afb2c2d5da65cf1add80560664b59d8ea64e6647a3c7c464a48cef8a0ccb1565" Sep 10 22:20:15.985553 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.985531 2576 scope.go:117] "RemoveContainer" containerID="afb2c2d5da65cf1add80560664b59d8ea64e6647a3c7c464a48cef8a0ccb1565" Sep 10 22:20:15.985885 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:20:15.985860 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afb2c2d5da65cf1add80560664b59d8ea64e6647a3c7c464a48cef8a0ccb1565\": container with ID starting with afb2c2d5da65cf1add80560664b59d8ea64e6647a3c7c464a48cef8a0ccb1565 not found: ID does not exist" containerID="afb2c2d5da65cf1add80560664b59d8ea64e6647a3c7c464a48cef8a0ccb1565" Sep 10 22:20:15.985995 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.985891 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afb2c2d5da65cf1add80560664b59d8ea64e6647a3c7c464a48cef8a0ccb1565"} err="failed to get container status \"afb2c2d5da65cf1add80560664b59d8ea64e6647a3c7c464a48cef8a0ccb1565\": rpc error: code = NotFound desc = could not find container \"afb2c2d5da65cf1add80560664b59d8ea64e6647a3c7c464a48cef8a0ccb1565\": container with ID starting with afb2c2d5da65cf1add80560664b59d8ea64e6647a3c7c464a48cef8a0ccb1565 not found: ID does not exist" Sep 10 22:20:15.993112 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.993083 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8"] Sep 10 22:20:15.996916 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:15.996889 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1c252b-7966db87b5-2ghb8"] Sep 10 22:20:16.007635 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:16.007604 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl"] Sep 10 22:20:16.011705 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:16.011677 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1c252b-7f9fc7446-dp4vl"] Sep 10 22:20:16.434046 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:16.434014 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6dc39f0-3ceb-4527-9b17-22cc8526d6c9" path="/var/lib/kubelet/pods/e6dc39f0-3ceb-4527-9b17-22cc8526d6c9/volumes" Sep 10 22:20:16.434454 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:16.434439 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8" path="/var/lib/kubelet/pods/fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8/volumes" Sep 10 22:20:17.067750 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:17.067719 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:17.293119 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:17.293088 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:18.036992 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:18.036956 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:18.434156 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:18.434115 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:18.853523 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:18.853481 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:21.735208 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:21.735163 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:21.847835 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:21.847774 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:22.297522 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:22.297494 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:22.385349 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:22.385313 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:22.539029 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:22.539000 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:22.941692 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:22.941656 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:23.050933 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.050901 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:23.139993 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.139957 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:23.245675 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.245568 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:23.533487 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.533410 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:23.712843 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.712783 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:23.845567 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.845535 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:23.864351 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864317 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975"] Sep 10 22:20:23.864710 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864699 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20953f3c-7e42-4683-8f19-ef7379cd2c04" containerName="payload-pre-processing" Sep 10 22:20:23.864759 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864714 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="20953f3c-7e42-4683-8f19-ef7379cd2c04" containerName="payload-pre-processing" Sep 10 22:20:23.864759 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864730 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb47e7b6-331c-41c5-8b24-3c1533b7fd09" containerName="payload-processing" Sep 10 22:20:23.864759 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864736 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb47e7b6-331c-41c5-8b24-3c1533b7fd09" containerName="payload-processing" Sep 10 22:20:23.864759 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864743 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8" containerName="payload-processing" Sep 10 22:20:23.864759 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864750 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8" containerName="payload-processing" Sep 10 22:20:23.864759 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864759 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72feb2e1-36b7-4a0f-b883-5f238cedecf5" containerName="maas-api" Sep 10 22:20:23.864963 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864766 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="72feb2e1-36b7-4a0f-b883-5f238cedecf5" containerName="maas-api" Sep 10 22:20:23.864963 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864773 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6dc39f0-3ceb-4527-9b17-22cc8526d6c9" containerName="payload-pre-processing" Sep 10 22:20:23.864963 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864778 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6dc39f0-3ceb-4527-9b17-22cc8526d6c9" containerName="payload-pre-processing" Sep 10 22:20:23.864963 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864787 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56c5bb6c-82e6-4eb6-bf62-036fc68ec72e" containerName="payload-processing" Sep 10 22:20:23.864963 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864792 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="56c5bb6c-82e6-4eb6-bf62-036fc68ec72e" containerName="payload-processing" Sep 10 22:20:23.864963 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864866 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="72feb2e1-36b7-4a0f-b883-5f238cedecf5" containerName="maas-api" Sep 10 22:20:23.864963 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864873 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6dc39f0-3ceb-4527-9b17-22cc8526d6c9" containerName="payload-pre-processing" Sep 10 22:20:23.864963 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864879 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="20953f3c-7e42-4683-8f19-ef7379cd2c04" containerName="payload-pre-processing" Sep 10 22:20:23.864963 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864884 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="56c5bb6c-82e6-4eb6-bf62-036fc68ec72e" containerName="payload-processing" Sep 10 22:20:23.864963 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864891 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb47e7b6-331c-41c5-8b24-3c1533b7fd09" containerName="payload-processing" Sep 10 22:20:23.864963 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.864896 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc070cc7-1ba8-4c20-a1e2-c9797f9b31a8" containerName="payload-processing" Sep 10 22:20:23.869410 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.869390 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" Sep 10 22:20:23.873664 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.873638 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-a9b6f6\"" Sep 10 22:20:23.873828 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.873703 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-a9b6f6-dockercfg-26zh9\"" Sep 10 22:20:23.898274 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.898242 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975"] Sep 10 22:20:23.901309 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.901275 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbcvv\" (UniqueName: \"kubernetes.io/projected/4425e040-737c-4a54-8b1d-f8627d949e74-kube-api-access-kbcvv\") pod \"payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975\" (UID: \"4425e040-737c-4a54-8b1d-f8627d949e74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" Sep 10 22:20:23.901479 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.901459 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4425e040-737c-4a54-8b1d-f8627d949e74-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975\" (UID: \"4425e040-737c-4a54-8b1d-f8627d949e74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" Sep 10 22:20:23.929878 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.929832 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp"] Sep 10 22:20:23.933415 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.933387 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" Sep 10 22:20:23.951131 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:23.951100 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp"] Sep 10 22:20:24.002011 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:24.001972 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4425e040-737c-4a54-8b1d-f8627d949e74-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975\" (UID: \"4425e040-737c-4a54-8b1d-f8627d949e74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" Sep 10 22:20:24.002011 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:24.002021 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmg2v\" (UniqueName: \"kubernetes.io/projected/df6e9305-12c1-4b84-98fa-c93328c2f4be-kube-api-access-tmg2v\") pod \"payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp\" (UID: \"df6e9305-12c1-4b84-98fa-c93328c2f4be\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" Sep 10 22:20:24.002274 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:24.002075 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df6e9305-12c1-4b84-98fa-c93328c2f4be-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp\" (UID: \"df6e9305-12c1-4b84-98fa-c93328c2f4be\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" Sep 10 22:20:24.002274 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:24.002104 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbcvv\" (UniqueName: \"kubernetes.io/projected/4425e040-737c-4a54-8b1d-f8627d949e74-kube-api-access-kbcvv\") pod \"payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975\" (UID: \"4425e040-737c-4a54-8b1d-f8627d949e74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" Sep 10 22:20:24.002791 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:24.002772 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4425e040-737c-4a54-8b1d-f8627d949e74-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975\" (UID: \"4425e040-737c-4a54-8b1d-f8627d949e74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" Sep 10 22:20:24.012453 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:24.012417 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbcvv\" (UniqueName: \"kubernetes.io/projected/4425e040-737c-4a54-8b1d-f8627d949e74-kube-api-access-kbcvv\") pod \"payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975\" (UID: \"4425e040-737c-4a54-8b1d-f8627d949e74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" Sep 10 22:20:24.103924 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:24.103820 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmg2v\" (UniqueName: \"kubernetes.io/projected/df6e9305-12c1-4b84-98fa-c93328c2f4be-kube-api-access-tmg2v\") pod \"payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp\" (UID: \"df6e9305-12c1-4b84-98fa-c93328c2f4be\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" Sep 10 22:20:24.103924 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:24.103918 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df6e9305-12c1-4b84-98fa-c93328c2f4be-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp\" (UID: \"df6e9305-12c1-4b84-98fa-c93328c2f4be\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" Sep 10 22:20:24.104718 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:24.104689 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df6e9305-12c1-4b84-98fa-c93328c2f4be-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp\" (UID: \"df6e9305-12c1-4b84-98fa-c93328c2f4be\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" Sep 10 22:20:24.115257 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:24.115224 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmg2v\" (UniqueName: \"kubernetes.io/projected/df6e9305-12c1-4b84-98fa-c93328c2f4be-kube-api-access-tmg2v\") pod \"payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp\" (UID: \"df6e9305-12c1-4b84-98fa-c93328c2f4be\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" Sep 10 22:20:24.179924 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:24.179879 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" Sep 10 22:20:24.245974 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:24.245939 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" Sep 10 22:20:24.463921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:24.463897 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975"] Sep 10 22:20:24.466380 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:20:24.466350 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4425e040_737c_4a54_8b1d_f8627d949e74.slice/crio-90f8d3575f47d2aad5620a9f8504b8c94db7936d0d9dc10bae9eded938be70bb WatchSource:0}: Error finding container 90f8d3575f47d2aad5620a9f8504b8c94db7936d0d9dc10bae9eded938be70bb: Status 404 returned error can't find the container with id 90f8d3575f47d2aad5620a9f8504b8c94db7936d0d9dc10bae9eded938be70bb Sep 10 22:20:24.535940 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:24.535904 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp"] Sep 10 22:20:24.538319 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:20:24.538289 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf6e9305_12c1_4b84_98fa_c93328c2f4be.slice/crio-11c9ac8ea69e2f4ea4b5b8fa836e7f935cb7daf9f9168f62ea30d0148a86944e WatchSource:0}: Error finding container 11c9ac8ea69e2f4ea4b5b8fa836e7f935cb7daf9f9168f62ea30d0148a86944e: Status 404 returned error can't find the container with id 11c9ac8ea69e2f4ea4b5b8fa836e7f935cb7daf9f9168f62ea30d0148a86944e Sep 10 22:20:25.022780 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:25.022743 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" event={"ID":"df6e9305-12c1-4b84-98fa-c93328c2f4be","Type":"ContainerStarted","Data":"7c50709fd2b1e356cb567df84a7b4fccc107f02a09c2e0aba183e90e3040defd"} Sep 10 22:20:25.023262 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:25.022834 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" event={"ID":"df6e9305-12c1-4b84-98fa-c93328c2f4be","Type":"ContainerStarted","Data":"11c9ac8ea69e2f4ea4b5b8fa836e7f935cb7daf9f9168f62ea30d0148a86944e"} Sep 10 22:20:25.023262 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:25.022871 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" Sep 10 22:20:25.024885 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:25.024862 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" event={"ID":"4425e040-737c-4a54-8b1d-f8627d949e74","Type":"ContainerStarted","Data":"996f2871c71a9b6921514877f1be1c2965c890f22f4cc4c11cc5438503a2132d"} Sep 10 22:20:25.025037 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:25.024899 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" event={"ID":"4425e040-737c-4a54-8b1d-f8627d949e74","Type":"ContainerStarted","Data":"90f8d3575f47d2aad5620a9f8504b8c94db7936d0d9dc10bae9eded938be70bb"} Sep 10 22:20:25.025037 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:25.025000 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" Sep 10 22:20:25.041788 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:25.041734 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" podStartSLOduration=2.041719407 podStartE2EDuration="2.041719407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:20:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:20:25.040194422 +0000 UTC m=+1155.161893231" watchObservedRunningTime="2026-09-10 22:20:25.041719407 +0000 UTC m=+1155.163418188" Sep 10 22:20:25.059482 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:25.059404 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" podStartSLOduration=2.059384478 podStartE2EDuration="2.059384478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:20:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:20:25.057006869 +0000 UTC m=+1155.178705654" watchObservedRunningTime="2026-09-10 22:20:25.059384478 +0000 UTC m=+1155.181083273" Sep 10 22:20:26.680899 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:26.680869 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:26.828306 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:26.828259 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:27.829904 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:27.829867 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf"] Sep 10 22:20:27.834347 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:27.834322 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:27.837526 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:27.837505 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-d07dfb-serving-cert\"" Sep 10 22:20:27.854784 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:27.854752 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf"] Sep 10 22:20:27.872124 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:27.872088 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:27.945060 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:27.945025 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:27.945234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:27.945168 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:27.945281 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:27.945235 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:27.945319 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:27.945291 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6266\" (UniqueName: \"kubernetes.io/projected/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-kube-api-access-x6266\") pod \"maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:27.970118 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:27.970080 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:28.046018 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.045977 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:28.046228 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.046036 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:28.046228 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.046091 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6266\" (UniqueName: \"kubernetes.io/projected/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-kube-api-access-x6266\") pod \"maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:28.046352 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.046281 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:28.046471 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:20:28.046427 2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-serving-cert: secret "maas-api-e2e-shared-a-w3-d07dfb-serving-cert" not found Sep 10 22:20:28.046518 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:20:28.046503 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-maas-api-tls podName:5e0a2a8f-1f81-4674-b8f9-dd2997485c8a nodeName:}" failed. No retries permitted until 2026-09-10 22:20:28.546483896 +0000 UTC m=+1158.668182680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-maas-api-tls") pod "maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" (UID: "5e0a2a8f-1f81-4674-b8f9-dd2997485c8a") : secret "maas-api-e2e-shared-a-w3-d07dfb-serving-cert" not found Sep 10 22:20:28.046746 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.046728 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:28.048547 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.048525 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:28.061783 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.061746 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6266\" (UniqueName: \"kubernetes.io/projected/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-kube-api-access-x6266\") pod \"maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:28.454875 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.454841 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:28.550408 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.550367 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:28.550624 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:20:28.550524 2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-serving-cert: secret "maas-api-e2e-shared-a-w3-d07dfb-serving-cert" not found Sep 10 22:20:28.550624 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:20:28.550598 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-maas-api-tls podName:5e0a2a8f-1f81-4674-b8f9-dd2997485c8a nodeName:}" failed. No retries permitted until 2026-09-10 22:20:29.550578063 +0000 UTC m=+1159.672276827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-maas-api-tls") pod "maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" (UID: "5e0a2a8f-1f81-4674-b8f9-dd2997485c8a") : secret "maas-api-e2e-shared-a-w3-d07dfb-serving-cert" not found Sep 10 22:20:28.552662 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.552635 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:28.645608 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.645571 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw"] Sep 10 22:20:28.649437 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.649410 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" Sep 10 22:20:28.651774 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.651754 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-d07dfb\"" Sep 10 22:20:28.653191 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.653176 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-d07dfb-dockercfg-bsmsc\"" Sep 10 22:20:28.668311 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.668281 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw"] Sep 10 22:20:28.753051 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.752959 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv4wq\" (UniqueName: \"kubernetes.io/projected/da7b2577-b37e-40f3-be88-047fb925454b-kube-api-access-vv4wq\") pod \"payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw\" (UID: \"da7b2577-b37e-40f3-be88-047fb925454b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" Sep 10 22:20:28.753051 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.752999 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da7b2577-b37e-40f3-be88-047fb925454b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw\" (UID: \"da7b2577-b37e-40f3-be88-047fb925454b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" Sep 10 22:20:28.853867 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.853828 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv4wq\" (UniqueName: \"kubernetes.io/projected/da7b2577-b37e-40f3-be88-047fb925454b-kube-api-access-vv4wq\") pod \"payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw\" (UID: \"da7b2577-b37e-40f3-be88-047fb925454b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" Sep 10 22:20:28.854241 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.853878 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da7b2577-b37e-40f3-be88-047fb925454b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw\" (UID: \"da7b2577-b37e-40f3-be88-047fb925454b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" Sep 10 22:20:28.854596 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.854574 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da7b2577-b37e-40f3-be88-047fb925454b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw\" (UID: \"da7b2577-b37e-40f3-be88-047fb925454b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" Sep 10 22:20:28.869466 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.869437 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv4wq\" (UniqueName: \"kubernetes.io/projected/da7b2577-b37e-40f3-be88-047fb925454b-kube-api-access-vv4wq\") pod \"payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw\" (UID: \"da7b2577-b37e-40f3-be88-047fb925454b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" Sep 10 22:20:28.880302 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.880260 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:28.960531 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:28.960490 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" Sep 10 22:20:29.251096 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:29.251063 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw"] Sep 10 22:20:29.252872 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:20:29.252845 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda7b2577_b37e_40f3_be88_047fb925454b.slice/crio-d84f5c6f8406fa9e6b07a235874818eea2595d3a1c5880eb49975b28d6497070 WatchSource:0}: Error finding container d84f5c6f8406fa9e6b07a235874818eea2595d3a1c5880eb49975b28d6497070: Status 404 returned error can't find the container with id d84f5c6f8406fa9e6b07a235874818eea2595d3a1c5880eb49975b28d6497070 Sep 10 22:20:29.561839 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:29.561712 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:29.564273 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:29.564246 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:29.647903 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:29.647863 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:29.946858 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:29.946821 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf"] Sep 10 22:20:29.948755 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:20:29.948729 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0a2a8f_1f81_4674_b8f9_dd2997485c8a.slice/crio-97e2710391e53424df3421f14bd29ffc597da41b564892efd419259ffb14c2f5 WatchSource:0}: Error finding container 97e2710391e53424df3421f14bd29ffc597da41b564892efd419259ffb14c2f5: Status 404 returned error can't find the container with id 97e2710391e53424df3421f14bd29ffc597da41b564892efd419259ffb14c2f5 Sep 10 22:20:29.950238 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:29.950219 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 22:20:30.052356 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:30.052315 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" event={"ID":"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a","Type":"ContainerStarted","Data":"97e2710391e53424df3421f14bd29ffc597da41b564892efd419259ffb14c2f5"} Sep 10 22:20:30.053915 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:30.053892 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" event={"ID":"da7b2577-b37e-40f3-be88-047fb925454b","Type":"ContainerStarted","Data":"5a3ca69641eab9038ac0e6dcc70dae2dd0c46619679ab95f36d990e10bc4a929"} Sep 10 22:20:30.054028 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:30.053934 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" event={"ID":"da7b2577-b37e-40f3-be88-047fb925454b","Type":"ContainerStarted","Data":"d84f5c6f8406fa9e6b07a235874818eea2595d3a1c5880eb49975b28d6497070"} Sep 10 22:20:30.054067 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:30.054038 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" Sep 10 22:20:30.698607 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:30.698518 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:31.060924 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:31.060836 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" event={"ID":"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a","Type":"ContainerStarted","Data":"c922636493540ac1b35b7526c5c9ce78af561d5238af42dd35fa7ae84525cc14"} Sep 10 22:20:31.060924 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:31.060907 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:31.084316 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:31.084260 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" podStartSLOduration=3.0842430690000002 podStartE2EDuration="3.084243069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:20:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:20:30.115711734 +0000 UTC m=+1160.237410516" watchObservedRunningTime="2026-09-10 22:20:31.084243069 +0000 UTC m=+1161.205941850" Sep 10 22:20:31.084901 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:31.084873 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" podStartSLOduration=3.691442943 podStartE2EDuration="4.084863658s" totalImagesPullingTime="393.420715ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:20:31.082110706 +0000 UTC m=+1161.203809491" watchObservedRunningTime="2026-09-10 22:20:31.084863658 +0000 UTC m=+1161.206562440" Sep 10 22:20:36.032902 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:36.032793 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" Sep 10 22:20:36.032902 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:36.032900 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" Sep 10 22:20:37.071626 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:37.071595 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:20:41.063726 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:41.063687 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" Sep 10 22:20:44.349485 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:44.349442 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:46.660040 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:46.660008 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:46.989401 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:46.989302 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:47.397513 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:47.397481 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:47.746718 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:47.746687 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:48.953185 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:48.953152 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:50.655095 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:50.655063 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:50.746691 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:50.746658 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:51.666064 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:51.666026 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:51.852500 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:51.852471 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:51.954962 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:51.954876 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:52.051876 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.051845 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:52.307513 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.307425 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:52.415415 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.415381 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:52.509506 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.509472 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:52.542636 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.542600 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g"] Sep 10 22:20:52.546306 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.546287 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" Sep 10 22:20:52.550114 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.550095 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-a9b3ec\"" Sep 10 22:20:52.558313 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.558251 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g"] Sep 10 22:20:52.605997 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.605961 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:52.692090 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.692055 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwkrx\" (UniqueName: \"kubernetes.io/projected/d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b-kube-api-access-gwkrx\") pod \"payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g\" (UID: \"d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" Sep 10 22:20:52.692579 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.692200 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g\" (UID: \"d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" Sep 10 22:20:52.793406 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.793364 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g\" (UID: \"d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" Sep 10 22:20:52.793589 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.793450 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwkrx\" (UniqueName: \"kubernetes.io/projected/d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b-kube-api-access-gwkrx\") pod \"payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g\" (UID: \"d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" Sep 10 22:20:52.794140 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.794118 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g\" (UID: \"d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" Sep 10 22:20:52.803475 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.803445 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwkrx\" (UniqueName: \"kubernetes.io/projected/d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b-kube-api-access-gwkrx\") pod \"payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g\" (UID: \"d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" Sep 10 22:20:52.858322 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:52.858283 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" Sep 10 22:20:53.140900 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:53.140872 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g"] Sep 10 22:20:53.143001 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:20:53.142975 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8fb0dd2_2f5a_407c_90d2_a6a4590b5c4b.slice/crio-41d22d60039cad87edd416c8b2f0c30812029bb1a29b1845ac0c8942d96fe7f5 WatchSource:0}: Error finding container 41d22d60039cad87edd416c8b2f0c30812029bb1a29b1845ac0c8942d96fe7f5: Status 404 returned error can't find the container with id 41d22d60039cad87edd416c8b2f0c30812029bb1a29b1845ac0c8942d96fe7f5 Sep 10 22:20:53.184183 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:53.184150 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" event={"ID":"d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b","Type":"ContainerStarted","Data":"41d22d60039cad87edd416c8b2f0c30812029bb1a29b1845ac0c8942d96fe7f5"} Sep 10 22:20:54.192986 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:54.192953 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" event={"ID":"d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b","Type":"ContainerStarted","Data":"6e2577fb6cad4f4f7d79b6f58fa1e600d3b03d49346f4a0a2e290b3e2a2b9bb7"} Sep 10 22:20:54.193394 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:54.193015 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" Sep 10 22:20:54.214468 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:54.214405 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" podStartSLOduration=2.214383086 podStartE2EDuration="2.214383086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:20:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:20:54.213577931 +0000 UTC m=+1184.335276734" watchObservedRunningTime="2026-09-10 22:20:54.214383086 +0000 UTC m=+1184.336081869" Sep 10 22:20:55.805966 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:55.805928 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:55.907353 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:55.907312 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:57.018613 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:57.018573 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:57.106336 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:57.106305 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:57.203693 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:57.203642 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:57.703093 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:57.703060 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:57.800630 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:57.800588 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:20:57.826435 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:57.826401 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762"] Sep 10 22:20:57.830361 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:57.830336 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" Sep 10 22:20:57.834920 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:57.834895 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-ec6d47\"" Sep 10 22:20:57.841645 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:57.841621 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c98cb902-7d04-4795-a410-97b27db8a0ce-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762\" (UID: \"c98cb902-7d04-4795-a410-97b27db8a0ce\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" Sep 10 22:20:57.841777 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:57.841665 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ltxp\" (UniqueName: \"kubernetes.io/projected/c98cb902-7d04-4795-a410-97b27db8a0ce-kube-api-access-5ltxp\") pod \"payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762\" (UID: \"c98cb902-7d04-4795-a410-97b27db8a0ce\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" Sep 10 22:20:57.848683 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:57.848649 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762"] Sep 10 22:20:57.942542 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:57.942496 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ltxp\" (UniqueName: \"kubernetes.io/projected/c98cb902-7d04-4795-a410-97b27db8a0ce-kube-api-access-5ltxp\") pod \"payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762\" (UID: \"c98cb902-7d04-4795-a410-97b27db8a0ce\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" Sep 10 22:20:57.942742 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:57.942612 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c98cb902-7d04-4795-a410-97b27db8a0ce-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762\" (UID: \"c98cb902-7d04-4795-a410-97b27db8a0ce\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" Sep 10 22:20:57.943331 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:57.943304 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c98cb902-7d04-4795-a410-97b27db8a0ce-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762\" (UID: \"c98cb902-7d04-4795-a410-97b27db8a0ce\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" Sep 10 22:20:57.951879 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:57.951844 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ltxp\" (UniqueName: \"kubernetes.io/projected/c98cb902-7d04-4795-a410-97b27db8a0ce-kube-api-access-5ltxp\") pod \"payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762\" (UID: \"c98cb902-7d04-4795-a410-97b27db8a0ce\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" Sep 10 22:20:58.141660 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:58.141621 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" Sep 10 22:20:58.425602 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:58.425570 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762"] Sep 10 22:20:58.428110 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:20:58.428066 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc98cb902_7d04_4795_a410_97b27db8a0ce.slice/crio-dfcef012bd2c72fd3dfdd8ee0c4eb34dd95ca642218ae3f86a9821c6aa330f5d WatchSource:0}: Error finding container dfcef012bd2c72fd3dfdd8ee0c4eb34dd95ca642218ae3f86a9821c6aa330f5d: Status 404 returned error can't find the container with id dfcef012bd2c72fd3dfdd8ee0c4eb34dd95ca642218ae3f86a9821c6aa330f5d Sep 10 22:20:59.227131 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:59.227090 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" event={"ID":"c98cb902-7d04-4795-a410-97b27db8a0ce","Type":"ContainerStarted","Data":"7468aa0358144eeecff92920a43c84f84c6e0a6e3cf203ae8ad1b6f750e9fec8"} Sep 10 22:20:59.227600 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:59.227157 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" Sep 10 22:20:59.227600 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:20:59.227175 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" event={"ID":"c98cb902-7d04-4795-a410-97b27db8a0ce","Type":"ContainerStarted","Data":"dfcef012bd2c72fd3dfdd8ee0c4eb34dd95ca642218ae3f86a9821c6aa330f5d"} Sep 10 22:21:05.200514 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:05.200476 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" Sep 10 22:21:05.219909 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:05.219845 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" podStartSLOduration=8.219827296 podStartE2EDuration="8.219827296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:20:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:20:59.267690102 +0000 UTC m=+1189.389388883" watchObservedRunningTime="2026-09-10 22:21:05.219827296 +0000 UTC m=+1195.341526102" Sep 10 22:21:10.234980 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:10.234950 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" Sep 10 22:21:12.994695 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:12.994639 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:15.456863 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:15.456828 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:18.156132 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:18.156090 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:20.757378 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:20.757326 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:24.633786 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:24.633750 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:24.727830 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:24.727770 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:26.953362 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:26.953316 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:27.287010 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:27.286911 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:27.498752 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:27.498717 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:30.769787 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:30.769759 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:30.869456 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:30.869422 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:32.111737 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:32.111703 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:32.212538 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:32.212505 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:32.626918 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:32.626883 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:32.726956 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:32.726922 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:32.827268 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:32.827238 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn"] Sep 10 22:21:32.831366 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:32.831343 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" Sep 10 22:21:32.833935 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:32.833910 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-676c26ab-dockercfg-n8vcv\"" Sep 10 22:21:32.834064 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:32.833972 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-676c26ab\"" Sep 10 22:21:32.848104 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:32.848075 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn"] Sep 10 22:21:32.899881 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:32.899777 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsdsd\" (UniqueName: \"kubernetes.io/projected/e4aa462f-5f09-4f13-93df-a81b91eb47b9-kube-api-access-jsdsd\") pod \"payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn\" (UID: \"e4aa462f-5f09-4f13-93df-a81b91eb47b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" Sep 10 22:21:32.900056 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:32.899905 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4aa462f-5f09-4f13-93df-a81b91eb47b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn\" (UID: \"e4aa462f-5f09-4f13-93df-a81b91eb47b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" Sep 10 22:21:33.000927 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:33.000887 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4aa462f-5f09-4f13-93df-a81b91eb47b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn\" (UID: \"e4aa462f-5f09-4f13-93df-a81b91eb47b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" Sep 10 22:21:33.001122 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:33.001002 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsdsd\" (UniqueName: \"kubernetes.io/projected/e4aa462f-5f09-4f13-93df-a81b91eb47b9-kube-api-access-jsdsd\") pod \"payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn\" (UID: \"e4aa462f-5f09-4f13-93df-a81b91eb47b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" Sep 10 22:21:33.001680 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:33.001648 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4aa462f-5f09-4f13-93df-a81b91eb47b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn\" (UID: \"e4aa462f-5f09-4f13-93df-a81b91eb47b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" Sep 10 22:21:33.010310 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:33.010279 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsdsd\" (UniqueName: \"kubernetes.io/projected/e4aa462f-5f09-4f13-93df-a81b91eb47b9-kube-api-access-jsdsd\") pod \"payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn\" (UID: \"e4aa462f-5f09-4f13-93df-a81b91eb47b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" Sep 10 22:21:33.141637 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:33.141596 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" Sep 10 22:21:33.419530 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:33.419409 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn"] Sep 10 22:21:33.422215 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:21:33.422173 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4aa462f_5f09_4f13_93df_a81b91eb47b9.slice/crio-42cb720cc872dd2fa9fb09ed3a5ccb09340dca2a11dcdcd173c9bdebff042612 WatchSource:0}: Error finding container 42cb720cc872dd2fa9fb09ed3a5ccb09340dca2a11dcdcd173c9bdebff042612: Status 404 returned error can't find the container with id 42cb720cc872dd2fa9fb09ed3a5ccb09340dca2a11dcdcd173c9bdebff042612 Sep 10 22:21:34.413875 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:34.413838 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" event={"ID":"e4aa462f-5f09-4f13-93df-a81b91eb47b9","Type":"ContainerStarted","Data":"f4f236255abba1786d88321f87be77a1a78a9840c0fa0c6fc320e29f0c057bfe"} Sep 10 22:21:34.414260 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:34.413925 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" event={"ID":"e4aa462f-5f09-4f13-93df-a81b91eb47b9","Type":"ContainerStarted","Data":"42cb720cc872dd2fa9fb09ed3a5ccb09340dca2a11dcdcd173c9bdebff042612"} Sep 10 22:21:34.414260 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:34.413948 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" Sep 10 22:21:34.432427 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:34.432371 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" podStartSLOduration=2.432354819 podStartE2EDuration="2.432354819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:21:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:21:34.429318155 +0000 UTC m=+1224.551016939" watchObservedRunningTime="2026-09-10 22:21:34.432354819 +0000 UTC m=+1224.554053601" Sep 10 22:21:40.340137 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:40.340104 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:21:45.421638 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:45.421601 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" Sep 10 22:21:55.653553 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:21:55.653519 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:22:01.893846 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:22:01.893791 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:22:13.908693 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:22:13.908656 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:22:15.641207 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:22:15.641152 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:22:48.298714 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:22:48.298681 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:22:48.402267 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:22:48.402233 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:22:51.062468 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:22:51.062434 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:23:06.722549 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:06.722517 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:23:06.759250 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:06.759218 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn"] Sep 10 22:23:06.760080 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:06.760033 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" podUID="e4aa462f-5f09-4f13-93df-a81b91eb47b9" containerName="payload-pre-processing" containerID="cri-o://f4f236255abba1786d88321f87be77a1a78a9840c0fa0c6fc320e29f0c057bfe" gracePeriod=30 Sep 10 22:23:06.798579 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:06.798527 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:23:06.897426 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:06.897392 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:23:06.903927 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:06.903887 2576 generic.go:356] "Generic (PLEG): container finished" podID="e4aa462f-5f09-4f13-93df-a81b91eb47b9" containerID="f4f236255abba1786d88321f87be77a1a78a9840c0fa0c6fc320e29f0c057bfe" exitCode=0 Sep 10 22:23:06.904130 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:06.904082 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" event={"ID":"e4aa462f-5f09-4f13-93df-a81b91eb47b9","Type":"ContainerDied","Data":"f4f236255abba1786d88321f87be77a1a78a9840c0fa0c6fc320e29f0c057bfe"} Sep 10 22:23:06.962882 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:06.962851 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:23:07.024955 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:07.024927 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" Sep 10 22:23:07.162642 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:07.162607 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsdsd\" (UniqueName: \"kubernetes.io/projected/e4aa462f-5f09-4f13-93df-a81b91eb47b9-kube-api-access-jsdsd\") pod \"e4aa462f-5f09-4f13-93df-a81b91eb47b9\" (UID: \"e4aa462f-5f09-4f13-93df-a81b91eb47b9\") " Sep 10 22:23:07.162889 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:07.162710 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4aa462f-5f09-4f13-93df-a81b91eb47b9-plugins-config-volume\") pod \"e4aa462f-5f09-4f13-93df-a81b91eb47b9\" (UID: \"e4aa462f-5f09-4f13-93df-a81b91eb47b9\") " Sep 10 22:23:07.163131 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:07.163099 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4aa462f-5f09-4f13-93df-a81b91eb47b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e4aa462f-5f09-4f13-93df-a81b91eb47b9" (UID: "e4aa462f-5f09-4f13-93df-a81b91eb47b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:23:07.164924 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:07.164897 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4aa462f-5f09-4f13-93df-a81b91eb47b9-kube-api-access-jsdsd" (OuterVolumeSpecName: "kube-api-access-jsdsd") pod "e4aa462f-5f09-4f13-93df-a81b91eb47b9" (UID: "e4aa462f-5f09-4f13-93df-a81b91eb47b9"). InnerVolumeSpecName "kube-api-access-jsdsd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:23:07.264584 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:07.264492 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsdsd\" (UniqueName: \"kubernetes.io/projected/e4aa462f-5f09-4f13-93df-a81b91eb47b9-kube-api-access-jsdsd\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:23:07.264584 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:07.264526 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4aa462f-5f09-4f13-93df-a81b91eb47b9-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:23:07.910079 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:07.910051 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" Sep 10 22:23:07.910079 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:07.910057 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn" event={"ID":"e4aa462f-5f09-4f13-93df-a81b91eb47b9","Type":"ContainerDied","Data":"42cb720cc872dd2fa9fb09ed3a5ccb09340dca2a11dcdcd173c9bdebff042612"} Sep 10 22:23:07.910636 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:07.910127 2576 scope.go:117] "RemoveContainer" containerID="f4f236255abba1786d88321f87be77a1a78a9840c0fa0c6fc320e29f0c057bfe" Sep 10 22:23:07.933082 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:07.933052 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn"] Sep 10 22:23:07.937394 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:07.937367 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-676c26ab-5d549c8566-srzfn"] Sep 10 22:23:08.432375 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:08.432342 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4aa462f-5f09-4f13-93df-a81b91eb47b9" path="/var/lib/kubelet/pods/e4aa462f-5f09-4f13-93df-a81b91eb47b9/volumes" Sep 10 22:23:10.558266 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:10.558236 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:23:15.590522 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:15.590483 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:23:52.599573 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:52.599541 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:23:52.773847 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:52.773786 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:23:58.970632 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:58.970597 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:23:59.073345 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:59.073293 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:23:59.271586 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:23:59.271486 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:00.472027 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:00.471993 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:05.614810 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:05.614763 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:05.723577 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:05.723540 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:05.895873 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:05.895771 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:06.519921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:06.519884 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:06.606144 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:06.606111 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:06.706402 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:06.706373 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:07.907340 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:07.907297 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:13.965218 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:13.965179 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:14.066540 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:14.066504 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:14.658928 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:14.658890 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:18.570162 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:18.570126 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:18.673010 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:18.672978 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:19.357599 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:19.357570 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:20.474719 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:20.474677 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:20.575942 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:20.575907 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:20.674990 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:20.674956 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:24.855377 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:24.855344 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:25.920446 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:25.920414 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:31.245341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:31.245302 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:31.400090 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:31.400058 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:35.650239 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:35.650207 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:52.337288 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:52.337253 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:58.370643 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:58.370592 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:58.470755 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:58.470721 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:58.741884 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:58.741784 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:59.040112 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:59.040024 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:24:59.342424 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:24:59.342389 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:02.104484 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:02.104380 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:02.205044 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:02.205009 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:03.303624 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:03.303587 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:03.404231 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:03.404184 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:03.914893 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:03.914853 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:04.014571 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.014539 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:04.059680 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.059645 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh"] Sep 10 22:25:04.060121 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.060102 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4aa462f-5f09-4f13-93df-a81b91eb47b9" containerName="payload-pre-processing" Sep 10 22:25:04.060121 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.060118 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4aa462f-5f09-4f13-93df-a81b91eb47b9" containerName="payload-pre-processing" Sep 10 22:25:04.060330 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.060199 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4aa462f-5f09-4f13-93df-a81b91eb47b9" containerName="payload-pre-processing" Sep 10 22:25:04.063539 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.063516 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" Sep 10 22:25:04.066023 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.065996 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-9c0e33\"" Sep 10 22:25:04.066150 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.066125 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-9c0e33-dockercfg-5rd8z\"" Sep 10 22:25:04.093175 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.093144 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh"] Sep 10 22:25:04.132816 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.132765 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:04.245850 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.245749 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmg86\" (UniqueName: \"kubernetes.io/projected/f3b416ee-dc5a-47d5-847f-319f3a9b8be9-kube-api-access-rmg86\") pod \"payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh\" (UID: \"f3b416ee-dc5a-47d5-847f-319f3a9b8be9\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" Sep 10 22:25:04.245850 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.245841 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3b416ee-dc5a-47d5-847f-319f3a9b8be9-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh\" (UID: \"f3b416ee-dc5a-47d5-847f-319f3a9b8be9\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" Sep 10 22:25:04.347279 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.347244 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmg86\" (UniqueName: \"kubernetes.io/projected/f3b416ee-dc5a-47d5-847f-319f3a9b8be9-kube-api-access-rmg86\") pod \"payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh\" (UID: \"f3b416ee-dc5a-47d5-847f-319f3a9b8be9\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" Sep 10 22:25:04.347780 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.347313 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3b416ee-dc5a-47d5-847f-319f3a9b8be9-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh\" (UID: \"f3b416ee-dc5a-47d5-847f-319f3a9b8be9\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" Sep 10 22:25:04.348156 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.348136 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3b416ee-dc5a-47d5-847f-319f3a9b8be9-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh\" (UID: \"f3b416ee-dc5a-47d5-847f-319f3a9b8be9\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" Sep 10 22:25:04.358938 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.358903 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmg86\" (UniqueName: \"kubernetes.io/projected/f3b416ee-dc5a-47d5-847f-319f3a9b8be9-kube-api-access-rmg86\") pod \"payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh\" (UID: \"f3b416ee-dc5a-47d5-847f-319f3a9b8be9\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" Sep 10 22:25:04.374633 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.374577 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" Sep 10 22:25:04.679832 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:04.679787 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh"] Sep 10 22:25:04.682238 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:25:04.682211 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3b416ee_dc5a_47d5_847f_319f3a9b8be9.slice/crio-0e848b7ad566ba189fe3df89046ed50540c4f0e5d0cfe99f420e2eb08389ba13 WatchSource:0}: Error finding container 0e848b7ad566ba189fe3df89046ed50540c4f0e5d0cfe99f420e2eb08389ba13: Status 404 returned error can't find the container with id 0e848b7ad566ba189fe3df89046ed50540c4f0e5d0cfe99f420e2eb08389ba13 Sep 10 22:25:05.533658 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:05.533621 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" event={"ID":"f3b416ee-dc5a-47d5-847f-319f3a9b8be9","Type":"ContainerStarted","Data":"cf1b3c8180df2b7699bd8a988d28c543f0ea5070e058c59cae7e9f6c6616421d"} Sep 10 22:25:05.534093 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:05.533677 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" event={"ID":"f3b416ee-dc5a-47d5-847f-319f3a9b8be9","Type":"ContainerStarted","Data":"0e848b7ad566ba189fe3df89046ed50540c4f0e5d0cfe99f420e2eb08389ba13"} Sep 10 22:25:05.534093 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:05.533714 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" Sep 10 22:25:05.551522 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:05.551469 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" podStartSLOduration=1.551452001 podStartE2EDuration="1.551452001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:25:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:25:05.550290653 +0000 UTC m=+1435.671989438" watchObservedRunningTime="2026-09-10 22:25:05.551452001 +0000 UTC m=+1435.673150816" Sep 10 22:25:16.541019 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:16.540986 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" Sep 10 22:25:25.768532 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:25.768491 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:27.075897 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:27.075853 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:27.585442 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:27.585395 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:27.938123 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:27.938040 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:28.406966 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:28.406938 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:31.458221 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:31.458179 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:31.551679 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:31.551646 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:32.665120 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:32.665079 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:32.764757 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:32.764728 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:33.296969 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:33.296881 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:33.887042 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:33.887000 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:34.115530 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:34.115495 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:34.143731 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:34.143640 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z"] Sep 10 22:25:34.147867 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:34.147841 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" Sep 10 22:25:34.151869 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:34.151846 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-c142c5\"" Sep 10 22:25:34.152020 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:34.151853 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-c142c5-dockercfg-f4rn7\"" Sep 10 22:25:34.170287 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:34.170250 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z"] Sep 10 22:25:34.233730 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:34.233691 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djc9s\" (UniqueName: \"kubernetes.io/projected/f2d7a665-db32-4d60-be9d-fd8d2c31401f-kube-api-access-djc9s\") pod \"payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z\" (UID: \"f2d7a665-db32-4d60-be9d-fd8d2c31401f\") " pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" Sep 10 22:25:34.233943 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:34.233745 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2d7a665-db32-4d60-be9d-fd8d2c31401f-plugins-config-volume\") pod \"payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z\" (UID: \"f2d7a665-db32-4d60-be9d-fd8d2c31401f\") " pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" Sep 10 22:25:34.335277 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:34.335235 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djc9s\" (UniqueName: \"kubernetes.io/projected/f2d7a665-db32-4d60-be9d-fd8d2c31401f-kube-api-access-djc9s\") pod \"payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z\" (UID: \"f2d7a665-db32-4d60-be9d-fd8d2c31401f\") " pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" Sep 10 22:25:34.335458 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:34.335294 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2d7a665-db32-4d60-be9d-fd8d2c31401f-plugins-config-volume\") pod \"payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z\" (UID: \"f2d7a665-db32-4d60-be9d-fd8d2c31401f\") " pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" Sep 10 22:25:34.336059 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:34.336041 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2d7a665-db32-4d60-be9d-fd8d2c31401f-plugins-config-volume\") pod \"payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z\" (UID: \"f2d7a665-db32-4d60-be9d-fd8d2c31401f\") " pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" Sep 10 22:25:34.365176 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:34.365153 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djc9s\" (UniqueName: \"kubernetes.io/projected/f2d7a665-db32-4d60-be9d-fd8d2c31401f-kube-api-access-djc9s\") pod \"payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z\" (UID: \"f2d7a665-db32-4d60-be9d-fd8d2c31401f\") " pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" Sep 10 22:25:34.458937 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:34.458855 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" Sep 10 22:25:34.781958 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:34.781790 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z"] Sep 10 22:25:34.784880 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:25:34.784846 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2d7a665_db32_4d60_be9d_fd8d2c31401f.slice/crio-ad3860834a817c8cf78ba6468cedc7cd4f221dca0417f1b3121cd36b24d94372 WatchSource:0}: Error finding container ad3860834a817c8cf78ba6468cedc7cd4f221dca0417f1b3121cd36b24d94372: Status 404 returned error can't find the container with id ad3860834a817c8cf78ba6468cedc7cd4f221dca0417f1b3121cd36b24d94372 Sep 10 22:25:35.694224 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:35.694187 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" event={"ID":"f2d7a665-db32-4d60-be9d-fd8d2c31401f","Type":"ContainerStarted","Data":"5f8216ea43bf1d4b4aad92e325fe6b5d4e8bcf0179e94d3e02cb5d937f1d7330"} Sep 10 22:25:35.694722 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:35.694275 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" event={"ID":"f2d7a665-db32-4d60-be9d-fd8d2c31401f","Type":"ContainerStarted","Data":"ad3860834a817c8cf78ba6468cedc7cd4f221dca0417f1b3121cd36b24d94372"} Sep 10 22:25:35.694722 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:35.694324 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" Sep 10 22:25:35.735430 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:35.735377 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" podStartSLOduration=1.735359358 podStartE2EDuration="1.735359358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:25:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:25:35.732827231 +0000 UTC m=+1465.854526013" watchObservedRunningTime="2026-09-10 22:25:35.735359358 +0000 UTC m=+1465.857058146" Sep 10 22:25:45.465581 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:45.465540 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:46.702332 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:46.702301 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" Sep 10 22:25:50.059845 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:50.059786 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:25:55.005878 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:25:55.005838 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:01.423717 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:01.423676 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:04.489700 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:04.489664 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:14.121575 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:14.121541 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:19.007986 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:19.007953 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:19.796474 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:19.796429 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:19.826658 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:19.826620 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh"] Sep 10 22:26:19.826948 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:19.826920 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" podUID="f3b416ee-dc5a-47d5-847f-319f3a9b8be9" containerName="payload-processing" containerID="cri-o://cf1b3c8180df2b7699bd8a988d28c543f0ea5070e058c59cae7e9f6c6616421d" gracePeriod=30 Sep 10 22:26:19.896637 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:19.896603 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:19.951941 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:19.951904 2576 generic.go:356] "Generic (PLEG): container finished" podID="f3b416ee-dc5a-47d5-847f-319f3a9b8be9" containerID="cf1b3c8180df2b7699bd8a988d28c543f0ea5070e058c59cae7e9f6c6616421d" exitCode=0 Sep 10 22:26:19.952103 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:19.951977 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" event={"ID":"f3b416ee-dc5a-47d5-847f-319f3a9b8be9","Type":"ContainerDied","Data":"cf1b3c8180df2b7699bd8a988d28c543f0ea5070e058c59cae7e9f6c6616421d"} Sep 10 22:26:19.997018 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:19.996983 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:20.085174 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:20.085149 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" Sep 10 22:26:20.096209 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:20.096178 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:20.205329 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:20.205293 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3b416ee-dc5a-47d5-847f-319f3a9b8be9-plugins-config-volume\") pod \"f3b416ee-dc5a-47d5-847f-319f3a9b8be9\" (UID: \"f3b416ee-dc5a-47d5-847f-319f3a9b8be9\") " Sep 10 22:26:20.205513 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:20.205408 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmg86\" (UniqueName: \"kubernetes.io/projected/f3b416ee-dc5a-47d5-847f-319f3a9b8be9-kube-api-access-rmg86\") pod \"f3b416ee-dc5a-47d5-847f-319f3a9b8be9\" (UID: \"f3b416ee-dc5a-47d5-847f-319f3a9b8be9\") " Sep 10 22:26:20.205818 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:20.205771 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3b416ee-dc5a-47d5-847f-319f3a9b8be9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f3b416ee-dc5a-47d5-847f-319f3a9b8be9" (UID: "f3b416ee-dc5a-47d5-847f-319f3a9b8be9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:26:20.207566 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:20.207542 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3b416ee-dc5a-47d5-847f-319f3a9b8be9-kube-api-access-rmg86" (OuterVolumeSpecName: "kube-api-access-rmg86") pod "f3b416ee-dc5a-47d5-847f-319f3a9b8be9" (UID: "f3b416ee-dc5a-47d5-847f-319f3a9b8be9"). InnerVolumeSpecName "kube-api-access-rmg86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:26:20.306331 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:20.306288 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmg86\" (UniqueName: \"kubernetes.io/projected/f3b416ee-dc5a-47d5-847f-319f3a9b8be9-kube-api-access-rmg86\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:26:20.306331 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:20.306322 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3b416ee-dc5a-47d5-847f-319f3a9b8be9-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:26:20.358369 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:20.358334 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:20.959130 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:20.959101 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" Sep 10 22:26:20.959329 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:20.959134 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh" event={"ID":"f3b416ee-dc5a-47d5-847f-319f3a9b8be9","Type":"ContainerDied","Data":"0e848b7ad566ba189fe3df89046ed50540c4f0e5d0cfe99f420e2eb08389ba13"} Sep 10 22:26:20.959329 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:20.959209 2576 scope.go:117] "RemoveContainer" containerID="cf1b3c8180df2b7699bd8a988d28c543f0ea5070e058c59cae7e9f6c6616421d" Sep 10 22:26:20.976392 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:20.976355 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh"] Sep 10 22:26:20.978765 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:20.978737 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c0e33-9fc878c49-sk4nh"] Sep 10 22:26:22.432783 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:22.432749 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3b416ee-dc5a-47d5-847f-319f3a9b8be9" path="/var/lib/kubelet/pods/f3b416ee-dc5a-47d5-847f-319f3a9b8be9/volumes" Sep 10 22:26:26.224851 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:26.224818 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:26.324956 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:26.324921 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:26.483200 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:26.483126 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:27.115190 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:27.115155 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:27.217186 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:27.217153 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:28.191428 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:28.191400 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:33.515574 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:33.515478 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:42.509014 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:42.508964 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:42.545892 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:42.545858 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z"] Sep 10 22:26:42.546127 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:42.546084 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" podUID="f2d7a665-db32-4d60-be9d-fd8d2c31401f" containerName="payload-processing" containerID="cri-o://5f8216ea43bf1d4b4aad92e325fe6b5d4e8bcf0179e94d3e02cb5d937f1d7330" gracePeriod=30 Sep 10 22:26:42.610141 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:42.610103 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:42.709509 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:42.709477 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:42.810352 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:42.810318 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:42.819637 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:42.819606 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" Sep 10 22:26:42.937997 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:42.937963 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djc9s\" (UniqueName: \"kubernetes.io/projected/f2d7a665-db32-4d60-be9d-fd8d2c31401f-kube-api-access-djc9s\") pod \"f2d7a665-db32-4d60-be9d-fd8d2c31401f\" (UID: \"f2d7a665-db32-4d60-be9d-fd8d2c31401f\") " Sep 10 22:26:42.938213 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:42.938020 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2d7a665-db32-4d60-be9d-fd8d2c31401f-plugins-config-volume\") pod \"f2d7a665-db32-4d60-be9d-fd8d2c31401f\" (UID: \"f2d7a665-db32-4d60-be9d-fd8d2c31401f\") " Sep 10 22:26:42.938439 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:42.938411 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2d7a665-db32-4d60-be9d-fd8d2c31401f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f2d7a665-db32-4d60-be9d-fd8d2c31401f" (UID: "f2d7a665-db32-4d60-be9d-fd8d2c31401f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:26:42.940229 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:42.940193 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2d7a665-db32-4d60-be9d-fd8d2c31401f-kube-api-access-djc9s" (OuterVolumeSpecName: "kube-api-access-djc9s") pod "f2d7a665-db32-4d60-be9d-fd8d2c31401f" (UID: "f2d7a665-db32-4d60-be9d-fd8d2c31401f"). InnerVolumeSpecName "kube-api-access-djc9s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:26:43.039316 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:43.039268 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djc9s\" (UniqueName: \"kubernetes.io/projected/f2d7a665-db32-4d60-be9d-fd8d2c31401f-kube-api-access-djc9s\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:26:43.039316 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:43.039305 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2d7a665-db32-4d60-be9d-fd8d2c31401f-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:26:43.073951 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:43.073914 2576 generic.go:356] "Generic (PLEG): container finished" podID="f2d7a665-db32-4d60-be9d-fd8d2c31401f" containerID="5f8216ea43bf1d4b4aad92e325fe6b5d4e8bcf0179e94d3e02cb5d937f1d7330" exitCode=0 Sep 10 22:26:43.074130 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:43.073988 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" Sep 10 22:26:43.074130 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:43.073998 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" event={"ID":"f2d7a665-db32-4d60-be9d-fd8d2c31401f","Type":"ContainerDied","Data":"5f8216ea43bf1d4b4aad92e325fe6b5d4e8bcf0179e94d3e02cb5d937f1d7330"} Sep 10 22:26:43.074130 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:43.074077 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z" event={"ID":"f2d7a665-db32-4d60-be9d-fd8d2c31401f","Type":"ContainerDied","Data":"ad3860834a817c8cf78ba6468cedc7cd4f221dca0417f1b3121cd36b24d94372"} Sep 10 22:26:43.074130 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:43.074101 2576 scope.go:117] "RemoveContainer" containerID="5f8216ea43bf1d4b4aad92e325fe6b5d4e8bcf0179e94d3e02cb5d937f1d7330" Sep 10 22:26:43.083177 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:43.083159 2576 scope.go:117] "RemoveContainer" containerID="5f8216ea43bf1d4b4aad92e325fe6b5d4e8bcf0179e94d3e02cb5d937f1d7330" Sep 10 22:26:43.083540 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:26:43.083510 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f8216ea43bf1d4b4aad92e325fe6b5d4e8bcf0179e94d3e02cb5d937f1d7330\": container with ID starting with 5f8216ea43bf1d4b4aad92e325fe6b5d4e8bcf0179e94d3e02cb5d937f1d7330 not found: ID does not exist" containerID="5f8216ea43bf1d4b4aad92e325fe6b5d4e8bcf0179e94d3e02cb5d937f1d7330" Sep 10 22:26:43.083640 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:43.083550 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f8216ea43bf1d4b4aad92e325fe6b5d4e8bcf0179e94d3e02cb5d937f1d7330"} err="failed to get container status \"5f8216ea43bf1d4b4aad92e325fe6b5d4e8bcf0179e94d3e02cb5d937f1d7330\": rpc error: code = NotFound desc = could not find container \"5f8216ea43bf1d4b4aad92e325fe6b5d4e8bcf0179e94d3e02cb5d937f1d7330\": container with ID starting with 5f8216ea43bf1d4b4aad92e325fe6b5d4e8bcf0179e94d3e02cb5d937f1d7330 not found: ID does not exist" Sep 10 22:26:43.111666 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:43.111630 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z"] Sep 10 22:26:43.118304 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:43.118272 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c142c5-5674dbd8d5-rsg6z"] Sep 10 22:26:43.149090 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:43.149056 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:44.431955 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:44.431924 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2d7a665-db32-4d60-be9d-fd8d2c31401f" path="/var/lib/kubelet/pods/f2d7a665-db32-4d60-be9d-fd8d2c31401f/volumes" Sep 10 22:26:49.562142 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:49.562084 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:26:49.621474 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:26:49.621439 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:00.051615 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.051578 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:00.083457 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.083422 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp"] Sep 10 22:27:00.083718 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.083675 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" podUID="df6e9305-12c1-4b84-98fa-c93328c2f4be" containerName="payload-processing" containerID="cri-o://7c50709fd2b1e356cb567df84a7b4fccc107f02a09c2e0aba183e90e3040defd" gracePeriod=30 Sep 10 22:27:00.093260 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.093113 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975"] Sep 10 22:27:00.093361 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.093331 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" podUID="4425e040-737c-4a54-8b1d-f8627d949e74" containerName="payload-pre-processing" containerID="cri-o://996f2871c71a9b6921514877f1be1c2965c890f22f4cc4c11cc5438503a2132d" gracePeriod=30 Sep 10 22:27:00.127690 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.127656 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:00.190613 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.190577 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:00.275480 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.275451 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:00.349193 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.349161 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:00.352453 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.352428 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" Sep 10 22:27:00.381050 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.381021 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" Sep 10 22:27:00.410694 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.410657 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df6e9305-12c1-4b84-98fa-c93328c2f4be-plugins-config-volume\") pod \"df6e9305-12c1-4b84-98fa-c93328c2f4be\" (UID: \"df6e9305-12c1-4b84-98fa-c93328c2f4be\") " Sep 10 22:27:00.410886 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.410743 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4425e040-737c-4a54-8b1d-f8627d949e74-plugins-config-volume\") pod \"4425e040-737c-4a54-8b1d-f8627d949e74\" (UID: \"4425e040-737c-4a54-8b1d-f8627d949e74\") " Sep 10 22:27:00.410886 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.410871 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmg2v\" (UniqueName: \"kubernetes.io/projected/df6e9305-12c1-4b84-98fa-c93328c2f4be-kube-api-access-tmg2v\") pod \"df6e9305-12c1-4b84-98fa-c93328c2f4be\" (UID: \"df6e9305-12c1-4b84-98fa-c93328c2f4be\") " Sep 10 22:27:00.411168 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.411128 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df6e9305-12c1-4b84-98fa-c93328c2f4be-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "df6e9305-12c1-4b84-98fa-c93328c2f4be" (UID: "df6e9305-12c1-4b84-98fa-c93328c2f4be"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:27:00.411302 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.411177 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4425e040-737c-4a54-8b1d-f8627d949e74-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4425e040-737c-4a54-8b1d-f8627d949e74" (UID: "4425e040-737c-4a54-8b1d-f8627d949e74"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:27:00.413035 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.413007 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df6e9305-12c1-4b84-98fa-c93328c2f4be-kube-api-access-tmg2v" (OuterVolumeSpecName: "kube-api-access-tmg2v") pod "df6e9305-12c1-4b84-98fa-c93328c2f4be" (UID: "df6e9305-12c1-4b84-98fa-c93328c2f4be"). InnerVolumeSpecName "kube-api-access-tmg2v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:27:00.511638 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.511602 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbcvv\" (UniqueName: \"kubernetes.io/projected/4425e040-737c-4a54-8b1d-f8627d949e74-kube-api-access-kbcvv\") pod \"4425e040-737c-4a54-8b1d-f8627d949e74\" (UID: \"4425e040-737c-4a54-8b1d-f8627d949e74\") " Sep 10 22:27:00.512222 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.512062 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df6e9305-12c1-4b84-98fa-c93328c2f4be-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:27:00.512222 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.512090 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4425e040-737c-4a54-8b1d-f8627d949e74-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:27:00.512222 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.512100 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmg2v\" (UniqueName: \"kubernetes.io/projected/df6e9305-12c1-4b84-98fa-c93328c2f4be-kube-api-access-tmg2v\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:27:00.514038 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.514016 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4425e040-737c-4a54-8b1d-f8627d949e74-kube-api-access-kbcvv" (OuterVolumeSpecName: "kube-api-access-kbcvv") pod "4425e040-737c-4a54-8b1d-f8627d949e74" (UID: "4425e040-737c-4a54-8b1d-f8627d949e74"). InnerVolumeSpecName "kube-api-access-kbcvv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:27:00.613240 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.613193 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbcvv\" (UniqueName: \"kubernetes.io/projected/4425e040-737c-4a54-8b1d-f8627d949e74-kube-api-access-kbcvv\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:27:00.622995 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:00.622959 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:01.167701 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.167664 2576 generic.go:356] "Generic (PLEG): container finished" podID="df6e9305-12c1-4b84-98fa-c93328c2f4be" containerID="7c50709fd2b1e356cb567df84a7b4fccc107f02a09c2e0aba183e90e3040defd" exitCode=0 Sep 10 22:27:01.168159 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.167734 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" Sep 10 22:27:01.168159 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.167747 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" event={"ID":"df6e9305-12c1-4b84-98fa-c93328c2f4be","Type":"ContainerDied","Data":"7c50709fd2b1e356cb567df84a7b4fccc107f02a09c2e0aba183e90e3040defd"} Sep 10 22:27:01.168159 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.167873 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp" event={"ID":"df6e9305-12c1-4b84-98fa-c93328c2f4be","Type":"ContainerDied","Data":"11c9ac8ea69e2f4ea4b5b8fa836e7f935cb7daf9f9168f62ea30d0148a86944e"} Sep 10 22:27:01.168159 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.167891 2576 scope.go:117] "RemoveContainer" containerID="7c50709fd2b1e356cb567df84a7b4fccc107f02a09c2e0aba183e90e3040defd" Sep 10 22:27:01.169371 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.169332 2576 generic.go:356] "Generic (PLEG): container finished" podID="4425e040-737c-4a54-8b1d-f8627d949e74" containerID="996f2871c71a9b6921514877f1be1c2965c890f22f4cc4c11cc5438503a2132d" exitCode=0 Sep 10 22:27:01.169448 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.169428 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" Sep 10 22:27:01.169503 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.169452 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" event={"ID":"4425e040-737c-4a54-8b1d-f8627d949e74","Type":"ContainerDied","Data":"996f2871c71a9b6921514877f1be1c2965c890f22f4cc4c11cc5438503a2132d"} Sep 10 22:27:01.169503 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.169492 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975" event={"ID":"4425e040-737c-4a54-8b1d-f8627d949e74","Type":"ContainerDied","Data":"90f8d3575f47d2aad5620a9f8504b8c94db7936d0d9dc10bae9eded938be70bb"} Sep 10 22:27:01.178411 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.178377 2576 scope.go:117] "RemoveContainer" containerID="7c50709fd2b1e356cb567df84a7b4fccc107f02a09c2e0aba183e90e3040defd" Sep 10 22:27:01.178774 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:27:01.178751 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c50709fd2b1e356cb567df84a7b4fccc107f02a09c2e0aba183e90e3040defd\": container with ID starting with 7c50709fd2b1e356cb567df84a7b4fccc107f02a09c2e0aba183e90e3040defd not found: ID does not exist" containerID="7c50709fd2b1e356cb567df84a7b4fccc107f02a09c2e0aba183e90e3040defd" Sep 10 22:27:01.178875 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.178785 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c50709fd2b1e356cb567df84a7b4fccc107f02a09c2e0aba183e90e3040defd"} err="failed to get container status \"7c50709fd2b1e356cb567df84a7b4fccc107f02a09c2e0aba183e90e3040defd\": rpc error: code = NotFound desc = could not find container \"7c50709fd2b1e356cb567df84a7b4fccc107f02a09c2e0aba183e90e3040defd\": container with ID starting with 7c50709fd2b1e356cb567df84a7b4fccc107f02a09c2e0aba183e90e3040defd not found: ID does not exist" Sep 10 22:27:01.178875 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.178831 2576 scope.go:117] "RemoveContainer" containerID="996f2871c71a9b6921514877f1be1c2965c890f22f4cc4c11cc5438503a2132d" Sep 10 22:27:01.189624 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.189603 2576 scope.go:117] "RemoveContainer" containerID="996f2871c71a9b6921514877f1be1c2965c890f22f4cc4c11cc5438503a2132d" Sep 10 22:27:01.190060 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:27:01.190035 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"996f2871c71a9b6921514877f1be1c2965c890f22f4cc4c11cc5438503a2132d\": container with ID starting with 996f2871c71a9b6921514877f1be1c2965c890f22f4cc4c11cc5438503a2132d not found: ID does not exist" containerID="996f2871c71a9b6921514877f1be1c2965c890f22f4cc4c11cc5438503a2132d" Sep 10 22:27:01.190178 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.190065 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"996f2871c71a9b6921514877f1be1c2965c890f22f4cc4c11cc5438503a2132d"} err="failed to get container status \"996f2871c71a9b6921514877f1be1c2965c890f22f4cc4c11cc5438503a2132d\": rpc error: code = NotFound desc = could not find container \"996f2871c71a9b6921514877f1be1c2965c890f22f4cc4c11cc5438503a2132d\": container with ID starting with 996f2871c71a9b6921514877f1be1c2965c890f22f4cc4c11cc5438503a2132d not found: ID does not exist" Sep 10 22:27:01.190543 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.190526 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp"] Sep 10 22:27:01.193869 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.193838 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a9b6f6-7db9db4c65-znclp"] Sep 10 22:27:01.206623 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.206583 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975"] Sep 10 22:27:01.208475 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:01.208452 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a9b6f6-fb5dff67d-8p975"] Sep 10 22:27:02.434329 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:02.434296 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4425e040-737c-4a54-8b1d-f8627d949e74" path="/var/lib/kubelet/pods/4425e040-737c-4a54-8b1d-f8627d949e74/volumes" Sep 10 22:27:02.434713 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:02.434660 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df6e9305-12c1-4b84-98fa-c93328c2f4be" path="/var/lib/kubelet/pods/df6e9305-12c1-4b84-98fa-c93328c2f4be/volumes" Sep 10 22:27:06.488094 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:06.488061 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:06.555539 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:06.555509 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:06.775914 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:06.775827 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:07.395631 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:07.395603 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:10.085590 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:10.085556 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:14.792921 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:14.792887 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:17.314485 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:17.314445 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:17.977299 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:17.977253 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g"] Sep 10 22:27:17.977593 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:17.977555 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" podUID="d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b" containerName="payload-processing" containerID="cri-o://6e2577fb6cad4f4f7d79b6f58fa1e600d3b03d49346f4a0a2e290b3e2a2b9bb7" gracePeriod=30 Sep 10 22:27:18.240642 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.240563 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" Sep 10 22:27:18.263115 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.263083 2576 generic.go:356] "Generic (PLEG): container finished" podID="d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b" containerID="6e2577fb6cad4f4f7d79b6f58fa1e600d3b03d49346f4a0a2e290b3e2a2b9bb7" exitCode=0 Sep 10 22:27:18.263306 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.263123 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" Sep 10 22:27:18.263306 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.263157 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" event={"ID":"d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b","Type":"ContainerDied","Data":"6e2577fb6cad4f4f7d79b6f58fa1e600d3b03d49346f4a0a2e290b3e2a2b9bb7"} Sep 10 22:27:18.263306 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.263210 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g" event={"ID":"d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b","Type":"ContainerDied","Data":"41d22d60039cad87edd416c8b2f0c30812029bb1a29b1845ac0c8942d96fe7f5"} Sep 10 22:27:18.263306 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.263236 2576 scope.go:117] "RemoveContainer" containerID="6e2577fb6cad4f4f7d79b6f58fa1e600d3b03d49346f4a0a2e290b3e2a2b9bb7" Sep 10 22:27:18.272853 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.272821 2576 scope.go:117] "RemoveContainer" containerID="6e2577fb6cad4f4f7d79b6f58fa1e600d3b03d49346f4a0a2e290b3e2a2b9bb7" Sep 10 22:27:18.273193 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:27:18.273172 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e2577fb6cad4f4f7d79b6f58fa1e600d3b03d49346f4a0a2e290b3e2a2b9bb7\": container with ID starting with 6e2577fb6cad4f4f7d79b6f58fa1e600d3b03d49346f4a0a2e290b3e2a2b9bb7 not found: ID does not exist" containerID="6e2577fb6cad4f4f7d79b6f58fa1e600d3b03d49346f4a0a2e290b3e2a2b9bb7" Sep 10 22:27:18.273293 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.273202 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e2577fb6cad4f4f7d79b6f58fa1e600d3b03d49346f4a0a2e290b3e2a2b9bb7"} err="failed to get container status \"6e2577fb6cad4f4f7d79b6f58fa1e600d3b03d49346f4a0a2e290b3e2a2b9bb7\": rpc error: code = NotFound desc = could not find container \"6e2577fb6cad4f4f7d79b6f58fa1e600d3b03d49346f4a0a2e290b3e2a2b9bb7\": container with ID starting with 6e2577fb6cad4f4f7d79b6f58fa1e600d3b03d49346f4a0a2e290b3e2a2b9bb7 not found: ID does not exist" Sep 10 22:27:18.393324 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.393290 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwkrx\" (UniqueName: \"kubernetes.io/projected/d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b-kube-api-access-gwkrx\") pod \"d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b\" (UID: \"d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b\") " Sep 10 22:27:18.393324 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.393326 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b-plugins-config-volume\") pod \"d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b\" (UID: \"d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b\") " Sep 10 22:27:18.393743 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.393715 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b" (UID: "d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:27:18.395558 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.395535 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b-kube-api-access-gwkrx" (OuterVolumeSpecName: "kube-api-access-gwkrx") pod "d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b" (UID: "d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b"). InnerVolumeSpecName "kube-api-access-gwkrx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:27:18.494575 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.494447 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwkrx\" (UniqueName: \"kubernetes.io/projected/d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b-kube-api-access-gwkrx\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:27:18.494575 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.494484 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:27:18.579825 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.579759 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g"] Sep 10 22:27:18.583808 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.583768 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a9b3ec-7dd4b487d5-2b82g"] Sep 10 22:27:18.701307 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:18.701273 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:20.431963 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:20.431925 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b" path="/var/lib/kubelet/pods/d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b/volumes" Sep 10 22:27:24.561576 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:24.561539 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:24.701786 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:24.701751 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:25.030914 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:25.030878 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:35.493066 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:35.493023 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:47.048081 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:47.048041 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:47.143438 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:47.143405 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:48.856936 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:48.856900 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:49.793234 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:49.793205 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:50.230935 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:50.230894 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:50.539553 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:50.539468 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:50.637090 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:50.637054 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:53.269225 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:53.269189 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:53.363483 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:53.363452 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:54.596015 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:54.595976 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:54.700601 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:54.700570 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:54.971341 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:54.971257 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:55.127783 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.127734 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:55.226911 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.226827 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:27:55.246004 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.245967 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn"] Sep 10 22:27:55.246421 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.246408 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2d7a665-db32-4d60-be9d-fd8d2c31401f" containerName="payload-processing" Sep 10 22:27:55.246472 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.246423 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d7a665-db32-4d60-be9d-fd8d2c31401f" containerName="payload-processing" Sep 10 22:27:55.246472 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.246438 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b" containerName="payload-processing" Sep 10 22:27:55.246472 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.246443 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b" containerName="payload-processing" Sep 10 22:27:55.246472 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.246454 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4425e040-737c-4a54-8b1d-f8627d949e74" containerName="payload-pre-processing" Sep 10 22:27:55.246472 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.246463 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4425e040-737c-4a54-8b1d-f8627d949e74" containerName="payload-pre-processing" Sep 10 22:27:55.246621 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.246474 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df6e9305-12c1-4b84-98fa-c93328c2f4be" containerName="payload-processing" Sep 10 22:27:55.246621 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.246481 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="df6e9305-12c1-4b84-98fa-c93328c2f4be" containerName="payload-processing" Sep 10 22:27:55.246621 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.246487 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3b416ee-dc5a-47d5-847f-319f3a9b8be9" containerName="payload-processing" Sep 10 22:27:55.246621 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.246492 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3b416ee-dc5a-47d5-847f-319f3a9b8be9" containerName="payload-processing" Sep 10 22:27:55.246621 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.246556 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4425e040-737c-4a54-8b1d-f8627d949e74" containerName="payload-pre-processing" Sep 10 22:27:55.246621 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.246565 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="df6e9305-12c1-4b84-98fa-c93328c2f4be" containerName="payload-processing" Sep 10 22:27:55.246621 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.246571 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8fb0dd2-2f5a-407c-90d2-a6a4590b5c4b" containerName="payload-processing" Sep 10 22:27:55.246621 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.246579 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2d7a665-db32-4d60-be9d-fd8d2c31401f" containerName="payload-processing" Sep 10 22:27:55.246621 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.246589 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3b416ee-dc5a-47d5-847f-319f3a9b8be9" containerName="payload-processing" Sep 10 22:27:55.249855 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.249837 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" Sep 10 22:27:55.251985 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.251960 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-a12859\"" Sep 10 22:27:55.260434 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.260405 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn"] Sep 10 22:27:55.265569 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.265542 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl"] Sep 10 22:27:55.268977 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.268954 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" Sep 10 22:27:55.271060 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.271037 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-a12859-dockercfg-mtl8q\"" Sep 10 22:27:55.282558 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.282527 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl"] Sep 10 22:27:55.360130 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.360090 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbl57\" (UniqueName: \"kubernetes.io/projected/c0e082a2-5efd-43af-b111-bdaf33e210e3-kube-api-access-wbl57\") pod \"payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn\" (UID: \"c0e082a2-5efd-43af-b111-bdaf33e210e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" Sep 10 22:27:55.360381 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.360326 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0b1baa2-6e96-4c02-a4fd-0ec5718995ef-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl\" (UID: \"c0b1baa2-6e96-4c02-a4fd-0ec5718995ef\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" Sep 10 22:27:55.360469 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.360420 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzv9n\" (UniqueName: \"kubernetes.io/projected/c0b1baa2-6e96-4c02-a4fd-0ec5718995ef-kube-api-access-pzv9n\") pod \"payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl\" (UID: \"c0b1baa2-6e96-4c02-a4fd-0ec5718995ef\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" Sep 10 22:27:55.360615 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.360496 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e082a2-5efd-43af-b111-bdaf33e210e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn\" (UID: \"c0e082a2-5efd-43af-b111-bdaf33e210e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" Sep 10 22:27:55.461336 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.461290 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzv9n\" (UniqueName: \"kubernetes.io/projected/c0b1baa2-6e96-4c02-a4fd-0ec5718995ef-kube-api-access-pzv9n\") pod \"payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl\" (UID: \"c0b1baa2-6e96-4c02-a4fd-0ec5718995ef\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" Sep 10 22:27:55.461524 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.461371 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e082a2-5efd-43af-b111-bdaf33e210e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn\" (UID: \"c0e082a2-5efd-43af-b111-bdaf33e210e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" Sep 10 22:27:55.461524 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.461443 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbl57\" (UniqueName: \"kubernetes.io/projected/c0e082a2-5efd-43af-b111-bdaf33e210e3-kube-api-access-wbl57\") pod \"payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn\" (UID: \"c0e082a2-5efd-43af-b111-bdaf33e210e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" Sep 10 22:27:55.461524 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.461493 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0b1baa2-6e96-4c02-a4fd-0ec5718995ef-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl\" (UID: \"c0b1baa2-6e96-4c02-a4fd-0ec5718995ef\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" Sep 10 22:27:55.462156 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.462135 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e082a2-5efd-43af-b111-bdaf33e210e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn\" (UID: \"c0e082a2-5efd-43af-b111-bdaf33e210e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" Sep 10 22:27:55.462241 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.462224 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0b1baa2-6e96-4c02-a4fd-0ec5718995ef-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl\" (UID: \"c0b1baa2-6e96-4c02-a4fd-0ec5718995ef\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" Sep 10 22:27:55.469729 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.469701 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbl57\" (UniqueName: \"kubernetes.io/projected/c0e082a2-5efd-43af-b111-bdaf33e210e3-kube-api-access-wbl57\") pod \"payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn\" (UID: \"c0e082a2-5efd-43af-b111-bdaf33e210e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" Sep 10 22:27:55.469896 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.469708 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzv9n\" (UniqueName: \"kubernetes.io/projected/c0b1baa2-6e96-4c02-a4fd-0ec5718995ef-kube-api-access-pzv9n\") pod \"payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl\" (UID: \"c0b1baa2-6e96-4c02-a4fd-0ec5718995ef\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" Sep 10 22:27:55.562372 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.562266 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" Sep 10 22:27:55.582471 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.582431 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" Sep 10 22:27:55.841177 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.841153 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn"] Sep 10 22:27:55.844001 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:27:55.843966 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0e082a2_5efd_43af_b111_bdaf33e210e3.slice/crio-7ee4cf28ce2e22aa5d079bcf9b3a936e80fd5feed9c977026e287cbca7d9c58c WatchSource:0}: Error finding container 7ee4cf28ce2e22aa5d079bcf9b3a936e80fd5feed9c977026e287cbca7d9c58c: Status 404 returned error can't find the container with id 7ee4cf28ce2e22aa5d079bcf9b3a936e80fd5feed9c977026e287cbca7d9c58c Sep 10 22:27:55.871049 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:55.871025 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl"] Sep 10 22:27:55.877012 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:27:55.876986 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0b1baa2_6e96_4c02_a4fd_0ec5718995ef.slice/crio-8d79e8998ed21a9c0841bee38aee9fd215d2a59eb55baabb0e26f29816e2d52a WatchSource:0}: Error finding container 8d79e8998ed21a9c0841bee38aee9fd215d2a59eb55baabb0e26f29816e2d52a: Status 404 returned error can't find the container with id 8d79e8998ed21a9c0841bee38aee9fd215d2a59eb55baabb0e26f29816e2d52a Sep 10 22:27:56.454946 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:56.454898 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" event={"ID":"c0b1baa2-6e96-4c02-a4fd-0ec5718995ef","Type":"ContainerStarted","Data":"bb967a5dbcb2157a5595ee5f7f6f165407f14c507288929184cff200cdb9410f"} Sep 10 22:27:56.455163 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:56.454984 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" Sep 10 22:27:56.455163 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:56.455011 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" event={"ID":"c0b1baa2-6e96-4c02-a4fd-0ec5718995ef","Type":"ContainerStarted","Data":"8d79e8998ed21a9c0841bee38aee9fd215d2a59eb55baabb0e26f29816e2d52a"} Sep 10 22:27:56.456521 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:56.456492 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" event={"ID":"c0e082a2-5efd-43af-b111-bdaf33e210e3","Type":"ContainerStarted","Data":"9cfab62c11a02085b7e4c2ee57cb686edc60f8e081c53af5b3b0c7e026f06800"} Sep 10 22:27:56.456670 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:56.456549 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" Sep 10 22:27:56.456670 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:56.456573 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" event={"ID":"c0e082a2-5efd-43af-b111-bdaf33e210e3","Type":"ContainerStarted","Data":"7ee4cf28ce2e22aa5d079bcf9b3a936e80fd5feed9c977026e287cbca7d9c58c"} Sep 10 22:27:56.476472 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:56.476413 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" podStartSLOduration=1.476395093 podStartE2EDuration="1.476395093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:27:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:27:56.473312489 +0000 UTC m=+1606.595011273" watchObservedRunningTime="2026-09-10 22:27:56.476395093 +0000 UTC m=+1606.598093886" Sep 10 22:27:56.488792 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:27:56.488714 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" podStartSLOduration=1.48867676 podStartE2EDuration="1.48867676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:27:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:27:56.488195883 +0000 UTC m=+1606.609894665" watchObservedRunningTime="2026-09-10 22:27:56.48867676 +0000 UTC m=+1606.610375543" Sep 10 22:28:04.417275 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:04.417198 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:28:07.464981 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:07.464949 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" Sep 10 22:28:07.465439 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:07.465020 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" Sep 10 22:28:08.293811 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:08.293762 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:28:08.394818 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:08.394780 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:28:18.176340 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:18.176305 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:28:18.834576 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:18.834542 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:28:19.159409 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:19.159311 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:28:19.275111 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:19.275077 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:28:22.041303 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:22.041261 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:28:22.135290 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:22.135256 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:28:23.321549 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:23.321518 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:28:23.406635 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:23.406597 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:28:23.854968 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:23.854940 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:28:23.954852 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:23.954789 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:28:23.995058 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:23.995014 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn"] Sep 10 22:28:24.005849 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:24.005813 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" Sep 10 22:28:24.008065 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:24.008036 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn"] Sep 10 22:28:24.008585 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:24.008564 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-79434a\"" Sep 10 22:28:24.008771 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:24.008566 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-79434a-dockercfg-prg7w\"" Sep 10 22:28:24.148548 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:24.148452 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e77e3016-6c58-4fac-82ca-589a93dba0ad-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn\" (UID: \"e77e3016-6c58-4fac-82ca-589a93dba0ad\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" Sep 10 22:28:24.148548 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:24.148523 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm6v2\" (UniqueName: \"kubernetes.io/projected/e77e3016-6c58-4fac-82ca-589a93dba0ad-kube-api-access-gm6v2\") pod \"payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn\" (UID: \"e77e3016-6c58-4fac-82ca-589a93dba0ad\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" Sep 10 22:28:24.249853 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:24.249778 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm6v2\" (UniqueName: \"kubernetes.io/projected/e77e3016-6c58-4fac-82ca-589a93dba0ad-kube-api-access-gm6v2\") pod \"payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn\" (UID: \"e77e3016-6c58-4fac-82ca-589a93dba0ad\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" Sep 10 22:28:24.250039 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:24.249922 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e77e3016-6c58-4fac-82ca-589a93dba0ad-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn\" (UID: \"e77e3016-6c58-4fac-82ca-589a93dba0ad\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" Sep 10 22:28:24.250584 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:24.250564 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e77e3016-6c58-4fac-82ca-589a93dba0ad-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn\" (UID: \"e77e3016-6c58-4fac-82ca-589a93dba0ad\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" Sep 10 22:28:24.258592 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:24.258561 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm6v2\" (UniqueName: \"kubernetes.io/projected/e77e3016-6c58-4fac-82ca-589a93dba0ad-kube-api-access-gm6v2\") pod \"payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn\" (UID: \"e77e3016-6c58-4fac-82ca-589a93dba0ad\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" Sep 10 22:28:24.318939 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:24.318897 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" Sep 10 22:28:24.595193 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:24.595162 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn"] Sep 10 22:28:24.597249 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:28:24.597218 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode77e3016_6c58_4fac_82ca_589a93dba0ad.slice/crio-26efc34d27e56ffecf9230b351fcb25c3d917801109141a2f2f0e314a992756b WatchSource:0}: Error finding container 26efc34d27e56ffecf9230b351fcb25c3d917801109141a2f2f0e314a992756b: Status 404 returned error can't find the container with id 26efc34d27e56ffecf9230b351fcb25c3d917801109141a2f2f0e314a992756b Sep 10 22:28:24.613588 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:24.613551 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" event={"ID":"e77e3016-6c58-4fac-82ca-589a93dba0ad","Type":"ContainerStarted","Data":"26efc34d27e56ffecf9230b351fcb25c3d917801109141a2f2f0e314a992756b"} Sep 10 22:28:25.621067 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:25.621033 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" event={"ID":"e77e3016-6c58-4fac-82ca-589a93dba0ad","Type":"ContainerStarted","Data":"08b3daf6ff04a097339015cc9a3456efef779d7490d4171562057efe0422d639"} Sep 10 22:28:25.621577 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:25.621113 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" Sep 10 22:28:25.638691 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:25.638632 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" podStartSLOduration=2.638616169 podStartE2EDuration="2.638616169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:28:25.636457768 +0000 UTC m=+1635.758156551" watchObservedRunningTime="2026-09-10 22:28:25.638616169 +0000 UTC m=+1635.760315003" Sep 10 22:28:36.628781 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:36.628748 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" Sep 10 22:28:47.067596 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:47.067551 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:28:52.333409 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:28:52.333374 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn_e77e3016-6c58-4fac-82ca-589a93dba0ad/payload-processing/0.log" Sep 10 22:29:00.925519 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:00.925487 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:29:27.946022 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:27.945990 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl_c0b1baa2-6e96-4c02-a4fd-0ec5718995ef/payload-processing/0.log" Sep 10 22:29:28.189609 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:28.189581 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn_e77e3016-6c58-4fac-82ca-589a93dba0ad/payload-processing/0.log" Sep 10 22:29:28.457557 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:28.457527 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:29:28.887916 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:28.887877 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:29:29.417711 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:29.417676 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:29:31.843112 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:31.843075 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:29:31.928668 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:31.928634 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:29:33.061241 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:33.061205 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:29:33.161763 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:33.161730 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:29:33.400911 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:33.400842 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:29:33.598148 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:33.598103 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:29:33.704736 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:33.704617 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:29:33.932542 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:33.932507 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw"] Sep 10 22:29:33.936292 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:33.936273 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" Sep 10 22:29:33.939962 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:33.939936 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-996d51-dockercfg-7kc27\"" Sep 10 22:29:33.940740 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:33.940718 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-996d51\"" Sep 10 22:29:33.957323 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:33.957252 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw"] Sep 10 22:29:34.028850 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:34.028778 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5p7m\" (UniqueName: \"kubernetes.io/projected/a49e215a-af8d-43b5-b9cb-2bf8ab232da3-kube-api-access-b5p7m\") pod \"payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw\" (UID: \"a49e215a-af8d-43b5-b9cb-2bf8ab232da3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" Sep 10 22:29:34.029032 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:34.028930 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a49e215a-af8d-43b5-b9cb-2bf8ab232da3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw\" (UID: \"a49e215a-af8d-43b5-b9cb-2bf8ab232da3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" Sep 10 22:29:34.130468 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:34.130430 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a49e215a-af8d-43b5-b9cb-2bf8ab232da3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw\" (UID: \"a49e215a-af8d-43b5-b9cb-2bf8ab232da3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" Sep 10 22:29:34.130917 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:34.130510 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5p7m\" (UniqueName: \"kubernetes.io/projected/a49e215a-af8d-43b5-b9cb-2bf8ab232da3-kube-api-access-b5p7m\") pod \"payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw\" (UID: \"a49e215a-af8d-43b5-b9cb-2bf8ab232da3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" Sep 10 22:29:34.131243 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:34.131214 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a49e215a-af8d-43b5-b9cb-2bf8ab232da3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw\" (UID: \"a49e215a-af8d-43b5-b9cb-2bf8ab232da3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" Sep 10 22:29:34.149705 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:34.149672 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5p7m\" (UniqueName: \"kubernetes.io/projected/a49e215a-af8d-43b5-b9cb-2bf8ab232da3-kube-api-access-b5p7m\") pod \"payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw\" (UID: \"a49e215a-af8d-43b5-b9cb-2bf8ab232da3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" Sep 10 22:29:34.248485 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:34.248389 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" Sep 10 22:29:34.562157 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:34.562123 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw"] Sep 10 22:29:34.563470 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:29:34.563449 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda49e215a_af8d_43b5_b9cb_2bf8ab232da3.slice/crio-ba1e92ae3d214958da63e9ae10bd06b334cd080f9f42a9b2ff7f6299907612f2 WatchSource:0}: Error finding container ba1e92ae3d214958da63e9ae10bd06b334cd080f9f42a9b2ff7f6299907612f2: Status 404 returned error can't find the container with id ba1e92ae3d214958da63e9ae10bd06b334cd080f9f42a9b2ff7f6299907612f2 Sep 10 22:29:34.993114 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:34.993075 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" event={"ID":"a49e215a-af8d-43b5-b9cb-2bf8ab232da3","Type":"ContainerStarted","Data":"79a039ffce590cd0596f1b37c9ff025dcfd4aa253303a14ec4a68c639e84c407"} Sep 10 22:29:34.993326 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:34.993138 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" event={"ID":"a49e215a-af8d-43b5-b9cb-2bf8ab232da3","Type":"ContainerStarted","Data":"ba1e92ae3d214958da63e9ae10bd06b334cd080f9f42a9b2ff7f6299907612f2"} Sep 10 22:29:34.993326 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:34.993268 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" Sep 10 22:29:35.048884 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:35.048824 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" podStartSLOduration=2.048788438 podStartE2EDuration="2.048788438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:29:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:29:35.047551604 +0000 UTC m=+1705.169250399" watchObservedRunningTime="2026-09-10 22:29:35.048788438 +0000 UTC m=+1705.170487220" Sep 10 22:29:46.000899 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:46.000869 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" Sep 10 22:29:56.805815 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:29:56.805780 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:00.137773 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:00.137735 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817990-trvrg"] Sep 10 22:30:00.141779 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:00.141752 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d07dfb-29817990-tgb59"] Sep 10 22:30:00.141948 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:00.141893 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-trvrg" Sep 10 22:30:00.144632 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:00.144608 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hjrv\"" Sep 10 22:30:00.145515 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:00.145491 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d07dfb-29817990-tgb59" Sep 10 22:30:00.151191 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:00.150836 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817990-trvrg"] Sep 10 22:30:00.155604 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:00.155544 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d07dfb-29817990-tgb59"] Sep 10 22:30:00.158056 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:00.158030 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-trvrg" Sep 10 22:30:00.167488 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:00.167461 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d07dfb-29817990-tgb59" Sep 10 22:30:00.444585 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:00.444554 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817990-trvrg"] Sep 10 22:30:00.446702 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:30:00.446677 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c48036_3df4_4851_9819_308f0a33abb1.slice/crio-5d6f5005c718bde17ba40235fb130628c69dd8f0c29aac017109e385a9279241 WatchSource:0}: Error finding container 5d6f5005c718bde17ba40235fb130628c69dd8f0c29aac017109e385a9279241: Status 404 returned error can't find the container with id 5d6f5005c718bde17ba40235fb130628c69dd8f0c29aac017109e385a9279241 Sep 10 22:30:00.448892 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:00.448873 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 22:30:00.459727 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:00.459657 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d07dfb-29817990-tgb59"] Sep 10 22:30:00.462088 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:30:00.462059 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a36382_4699_4d59_8d6b_b392fea43829.slice/crio-7737937d0a42302773b91e67a16d5ac762fd25fa4b9bacd9b14ed176ed6a55d2 WatchSource:0}: Error finding container 7737937d0a42302773b91e67a16d5ac762fd25fa4b9bacd9b14ed176ed6a55d2: Status 404 returned error can't find the container with id 7737937d0a42302773b91e67a16d5ac762fd25fa4b9bacd9b14ed176ed6a55d2 Sep 10 22:30:01.131314 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:01.131276 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d07dfb-29817990-tgb59" event={"ID":"b8a36382-4699-4d59-8d6b-b392fea43829","Type":"ContainerStarted","Data":"7737937d0a42302773b91e67a16d5ac762fd25fa4b9bacd9b14ed176ed6a55d2"} Sep 10 22:30:01.132438 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:01.132414 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-trvrg" event={"ID":"24c48036-3df4-4851-9819-308f0a33abb1","Type":"ContainerStarted","Data":"5d6f5005c718bde17ba40235fb130628c69dd8f0c29aac017109e385a9279241"} Sep 10 22:30:03.149069 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:03.149029 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-trvrg" event={"ID":"24c48036-3df4-4851-9819-308f0a33abb1","Type":"ContainerStarted","Data":"910bbeeb2a0cfa410248842b8a03f490ee4e049c0258d6cad88fdb388e557c43"} Sep 10 22:30:03.150663 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:03.150629 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d07dfb-29817990-tgb59" event={"ID":"b8a36382-4699-4d59-8d6b-b392fea43829","Type":"ContainerStarted","Data":"4cf90d2d9e386c6b6f81231a1e78208f945a637f7ae0ade1d4546eedfffda72b"} Sep 10 22:30:03.163749 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:03.163684 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-trvrg" podStartSLOduration=0.847577415 podStartE2EDuration="3.163662341s" totalImagesPullingTime="2.316084926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:30:03.163213946 +0000 UTC m=+1733.284912729" watchObservedRunningTime="2026-09-10 22:30:03.163662341 +0000 UTC m=+1733.285361123" Sep 10 22:30:03.179123 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:03.179064 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d07dfb-29817990-tgb59" podStartSLOduration=0.873325474 podStartE2EDuration="3.179044514s" totalImagesPullingTime="2.30571904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:30:03.178252903 +0000 UTC m=+1733.299951688" watchObservedRunningTime="2026-09-10 22:30:03.179044514 +0000 UTC m=+1733.300743322" Sep 10 22:30:03.960097 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:03.959998 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:04.012249 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.012208 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw"] Sep 10 22:30:04.012547 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.012480 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" podUID="a49e215a-af8d-43b5-b9cb-2bf8ab232da3" containerName="payload-processing" containerID="cri-o://79a039ffce590cd0596f1b37c9ff025dcfd4aa253303a14ec4a68c639e84c407" gracePeriod=30 Sep 10 22:30:04.061943 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.061907 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:04.159556 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.159522 2576 generic.go:356] "Generic (PLEG): container finished" podID="a49e215a-af8d-43b5-b9cb-2bf8ab232da3" containerID="79a039ffce590cd0596f1b37c9ff025dcfd4aa253303a14ec4a68c639e84c407" exitCode=0 Sep 10 22:30:04.159556 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.159553 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:04.160193 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.159676 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" event={"ID":"a49e215a-af8d-43b5-b9cb-2bf8ab232da3","Type":"ContainerDied","Data":"79a039ffce590cd0596f1b37c9ff025dcfd4aa253303a14ec4a68c639e84c407"} Sep 10 22:30:04.161904 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.161873 2576 generic.go:356] "Generic (PLEG): container finished" podID="b8a36382-4699-4d59-8d6b-b392fea43829" containerID="4cf90d2d9e386c6b6f81231a1e78208f945a637f7ae0ade1d4546eedfffda72b" exitCode=0 Sep 10 22:30:04.162067 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.161954 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d07dfb-29817990-tgb59" event={"ID":"b8a36382-4699-4d59-8d6b-b392fea43829","Type":"ContainerDied","Data":"4cf90d2d9e386c6b6f81231a1e78208f945a637f7ae0ade1d4546eedfffda72b"} Sep 10 22:30:04.163831 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.163781 2576 generic.go:356] "Generic (PLEG): container finished" podID="24c48036-3df4-4851-9819-308f0a33abb1" containerID="910bbeeb2a0cfa410248842b8a03f490ee4e049c0258d6cad88fdb388e557c43" exitCode=0 Sep 10 22:30:04.163970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.163841 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-trvrg" event={"ID":"24c48036-3df4-4851-9819-308f0a33abb1","Type":"ContainerDied","Data":"910bbeeb2a0cfa410248842b8a03f490ee4e049c0258d6cad88fdb388e557c43"} Sep 10 22:30:04.261754 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.261674 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:04.286036 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.286010 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" Sep 10 22:30:04.345771 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.345735 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5p7m\" (UniqueName: \"kubernetes.io/projected/a49e215a-af8d-43b5-b9cb-2bf8ab232da3-kube-api-access-b5p7m\") pod \"a49e215a-af8d-43b5-b9cb-2bf8ab232da3\" (UID: \"a49e215a-af8d-43b5-b9cb-2bf8ab232da3\") " Sep 10 22:30:04.345997 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.345862 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a49e215a-af8d-43b5-b9cb-2bf8ab232da3-plugins-config-volume\") pod \"a49e215a-af8d-43b5-b9cb-2bf8ab232da3\" (UID: \"a49e215a-af8d-43b5-b9cb-2bf8ab232da3\") " Sep 10 22:30:04.346328 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.346293 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a49e215a-af8d-43b5-b9cb-2bf8ab232da3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a49e215a-af8d-43b5-b9cb-2bf8ab232da3" (UID: "a49e215a-af8d-43b5-b9cb-2bf8ab232da3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:30:04.346659 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.346637 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a49e215a-af8d-43b5-b9cb-2bf8ab232da3-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:30:04.357432 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.357392 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a49e215a-af8d-43b5-b9cb-2bf8ab232da3-kube-api-access-b5p7m" (OuterVolumeSpecName: "kube-api-access-b5p7m") pod "a49e215a-af8d-43b5-b9cb-2bf8ab232da3" (UID: "a49e215a-af8d-43b5-b9cb-2bf8ab232da3"). InnerVolumeSpecName "kube-api-access-b5p7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:30:04.447298 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.447268 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5p7m\" (UniqueName: \"kubernetes.io/projected/a49e215a-af8d-43b5-b9cb-2bf8ab232da3-kube-api-access-b5p7m\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:30:04.592887 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:04.592857 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:05.170745 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:05.170709 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" Sep 10 22:30:05.171278 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:05.170776 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw" event={"ID":"a49e215a-af8d-43b5-b9cb-2bf8ab232da3","Type":"ContainerDied","Data":"ba1e92ae3d214958da63e9ae10bd06b334cd080f9f42a9b2ff7f6299907612f2"} Sep 10 22:30:05.171278 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:05.170852 2576 scope.go:117] "RemoveContainer" containerID="79a039ffce590cd0596f1b37c9ff025dcfd4aa253303a14ec4a68c639e84c407" Sep 10 22:30:05.195572 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:05.195535 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw"] Sep 10 22:30:05.204829 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:05.204775 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-996d51-7877cd56d7-4c7nw"] Sep 10 22:30:05.323451 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:05.323421 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-trvrg" Sep 10 22:30:05.353875 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:05.353845 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d07dfb-29817990-tgb59" Sep 10 22:30:06.179014 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:06.178985 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d07dfb-29817990-tgb59" Sep 10 22:30:06.179014 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:06.179006 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d07dfb-29817990-tgb59" event={"ID":"b8a36382-4699-4d59-8d6b-b392fea43829","Type":"ContainerDied","Data":"7737937d0a42302773b91e67a16d5ac762fd25fa4b9bacd9b14ed176ed6a55d2"} Sep 10 22:30:06.179543 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:06.179054 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7737937d0a42302773b91e67a16d5ac762fd25fa4b9bacd9b14ed176ed6a55d2" Sep 10 22:30:06.180384 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:06.180358 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-trvrg" event={"ID":"24c48036-3df4-4851-9819-308f0a33abb1","Type":"ContainerDied","Data":"5d6f5005c718bde17ba40235fb130628c69dd8f0c29aac017109e385a9279241"} Sep 10 22:30:06.180384 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:06.180376 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-trvrg" Sep 10 22:30:06.180529 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:06.180402 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5d6f5005c718bde17ba40235fb130628c69dd8f0c29aac017109e385a9279241" Sep 10 22:30:06.432600 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:06.432521 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a49e215a-af8d-43b5-b9cb-2bf8ab232da3" path="/var/lib/kubelet/pods/a49e215a-af8d-43b5-b9cb-2bf8ab232da3/volumes" Sep 10 22:30:10.362548 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:10.362519 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:10.463419 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:10.463388 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:10.650569 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:10.650535 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:14.590370 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:14.590325 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:29.983889 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:29.983859 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:30.031964 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.031922 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl"] Sep 10 22:30:30.032283 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.032254 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" podUID="c0b1baa2-6e96-4c02-a4fd-0ec5718995ef" containerName="payload-processing" containerID="cri-o://bb967a5dbcb2157a5595ee5f7f6f165407f14c507288929184cff200cdb9410f" gracePeriod=30 Sep 10 22:30:30.041285 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.041237 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn"] Sep 10 22:30:30.041600 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.041575 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" podUID="c0e082a2-5efd-43af-b111-bdaf33e210e3" containerName="payload-pre-processing" containerID="cri-o://9cfab62c11a02085b7e4c2ee57cb686edc60f8e081c53af5b3b0c7e026f06800" gracePeriod=30 Sep 10 22:30:30.076589 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.076555 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:30.143774 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.143735 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:30.219560 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.219520 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:30.285200 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.285172 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:30.312861 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.312831 2576 generic.go:356] "Generic (PLEG): container finished" podID="c0b1baa2-6e96-4c02-a4fd-0ec5718995ef" containerID="bb967a5dbcb2157a5595ee5f7f6f165407f14c507288929184cff200cdb9410f" exitCode=0 Sep 10 22:30:30.313021 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.312890 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" event={"ID":"c0b1baa2-6e96-4c02-a4fd-0ec5718995ef","Type":"ContainerDied","Data":"bb967a5dbcb2157a5595ee5f7f6f165407f14c507288929184cff200cdb9410f"} Sep 10 22:30:30.313021 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.312972 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" event={"ID":"c0b1baa2-6e96-4c02-a4fd-0ec5718995ef","Type":"ContainerDied","Data":"8d79e8998ed21a9c0841bee38aee9fd215d2a59eb55baabb0e26f29816e2d52a"} Sep 10 22:30:30.313021 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.312990 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d79e8998ed21a9c0841bee38aee9fd215d2a59eb55baabb0e26f29816e2d52a" Sep 10 22:30:30.314351 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.314329 2576 generic.go:356] "Generic (PLEG): container finished" podID="c0e082a2-5efd-43af-b111-bdaf33e210e3" containerID="9cfab62c11a02085b7e4c2ee57cb686edc60f8e081c53af5b3b0c7e026f06800" exitCode=0 Sep 10 22:30:30.314462 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.314405 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" event={"ID":"c0e082a2-5efd-43af-b111-bdaf33e210e3","Type":"ContainerDied","Data":"9cfab62c11a02085b7e4c2ee57cb686edc60f8e081c53af5b3b0c7e026f06800"} Sep 10 22:30:30.326140 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.326109 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" Sep 10 22:30:30.329628 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.329605 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" Sep 10 22:30:30.513330 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.513234 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e082a2-5efd-43af-b111-bdaf33e210e3-plugins-config-volume\") pod \"c0e082a2-5efd-43af-b111-bdaf33e210e3\" (UID: \"c0e082a2-5efd-43af-b111-bdaf33e210e3\") " Sep 10 22:30:30.513495 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.513420 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzv9n\" (UniqueName: \"kubernetes.io/projected/c0b1baa2-6e96-4c02-a4fd-0ec5718995ef-kube-api-access-pzv9n\") pod \"c0b1baa2-6e96-4c02-a4fd-0ec5718995ef\" (UID: \"c0b1baa2-6e96-4c02-a4fd-0ec5718995ef\") " Sep 10 22:30:30.513495 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.513455 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbl57\" (UniqueName: \"kubernetes.io/projected/c0e082a2-5efd-43af-b111-bdaf33e210e3-kube-api-access-wbl57\") pod \"c0e082a2-5efd-43af-b111-bdaf33e210e3\" (UID: \"c0e082a2-5efd-43af-b111-bdaf33e210e3\") " Sep 10 22:30:30.513495 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.513483 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0b1baa2-6e96-4c02-a4fd-0ec5718995ef-plugins-config-volume\") pod \"c0b1baa2-6e96-4c02-a4fd-0ec5718995ef\" (UID: \"c0b1baa2-6e96-4c02-a4fd-0ec5718995ef\") " Sep 10 22:30:30.513718 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.513694 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0e082a2-5efd-43af-b111-bdaf33e210e3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c0e082a2-5efd-43af-b111-bdaf33e210e3" (UID: "c0e082a2-5efd-43af-b111-bdaf33e210e3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:30:30.513946 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.513921 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0b1baa2-6e96-4c02-a4fd-0ec5718995ef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c0b1baa2-6e96-4c02-a4fd-0ec5718995ef" (UID: "c0b1baa2-6e96-4c02-a4fd-0ec5718995ef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:30:30.515715 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.515689 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0e082a2-5efd-43af-b111-bdaf33e210e3-kube-api-access-wbl57" (OuterVolumeSpecName: "kube-api-access-wbl57") pod "c0e082a2-5efd-43af-b111-bdaf33e210e3" (UID: "c0e082a2-5efd-43af-b111-bdaf33e210e3"). InnerVolumeSpecName "kube-api-access-wbl57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:30:30.515826 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.515692 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0b1baa2-6e96-4c02-a4fd-0ec5718995ef-kube-api-access-pzv9n" (OuterVolumeSpecName: "kube-api-access-pzv9n") pod "c0b1baa2-6e96-4c02-a4fd-0ec5718995ef" (UID: "c0b1baa2-6e96-4c02-a4fd-0ec5718995ef"). InnerVolumeSpecName "kube-api-access-pzv9n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:30:30.589867 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.589784 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:30.615178 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.615138 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzv9n\" (UniqueName: \"kubernetes.io/projected/c0b1baa2-6e96-4c02-a4fd-0ec5718995ef-kube-api-access-pzv9n\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:30:30.615178 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.615179 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbl57\" (UniqueName: \"kubernetes.io/projected/c0e082a2-5efd-43af-b111-bdaf33e210e3-kube-api-access-wbl57\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:30:30.615178 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.615194 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0b1baa2-6e96-4c02-a4fd-0ec5718995ef-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:30:30.615456 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:30.615208 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e082a2-5efd-43af-b111-bdaf33e210e3-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:30:31.321072 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:31.321041 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" Sep 10 22:30:31.321679 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:31.321089 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn" event={"ID":"c0e082a2-5efd-43af-b111-bdaf33e210e3","Type":"ContainerDied","Data":"7ee4cf28ce2e22aa5d079bcf9b3a936e80fd5feed9c977026e287cbca7d9c58c"} Sep 10 22:30:31.321679 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:31.321148 2576 scope.go:117] "RemoveContainer" containerID="9cfab62c11a02085b7e4c2ee57cb686edc60f8e081c53af5b3b0c7e026f06800" Sep 10 22:30:31.321679 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:31.321281 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl" Sep 10 22:30:31.347697 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:31.347664 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl"] Sep 10 22:30:31.351231 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:31.351202 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a12859-97656985c-jhgjl"] Sep 10 22:30:31.360200 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:31.360166 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn"] Sep 10 22:30:31.366529 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:31.366496 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a12859-7ff8d47b86-qpkbn"] Sep 10 22:30:32.433006 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:32.432973 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0b1baa2-6e96-4c02-a4fd-0ec5718995ef" path="/var/lib/kubelet/pods/c0b1baa2-6e96-4c02-a4fd-0ec5718995ef/volumes" Sep 10 22:30:32.433625 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:32.433515 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0e082a2-5efd-43af-b111-bdaf33e210e3" path="/var/lib/kubelet/pods/c0e082a2-5efd-43af-b111-bdaf33e210e3/volumes" Sep 10 22:30:35.666345 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:35.666312 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:30:50.922942 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:30:50.922911 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:11.991151 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:11.991112 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:13.151145 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:13.151110 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:18.710535 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:18.710500 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:18.812317 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:18.812281 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:18.984396 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:18.984291 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:19.581114 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:19.581072 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:20.484936 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:20.484901 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:21.186516 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:21.186482 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:29.994280 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:29.994247 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:30.034770 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.034726 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn"] Sep 10 22:31:30.035054 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.035022 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" podUID="e77e3016-6c58-4fac-82ca-589a93dba0ad" containerName="payload-processing" containerID="cri-o://08b3daf6ff04a097339015cc9a3456efef779d7490d4171562057efe0422d639" gracePeriod=30 Sep 10 22:31:30.094157 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.094125 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:30.194325 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.194289 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:30.294580 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.294548 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:30.298937 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.298904 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" Sep 10 22:31:30.398264 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.398225 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e77e3016-6c58-4fac-82ca-589a93dba0ad-plugins-config-volume\") pod \"e77e3016-6c58-4fac-82ca-589a93dba0ad\" (UID: \"e77e3016-6c58-4fac-82ca-589a93dba0ad\") " Sep 10 22:31:30.398436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.398310 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm6v2\" (UniqueName: \"kubernetes.io/projected/e77e3016-6c58-4fac-82ca-589a93dba0ad-kube-api-access-gm6v2\") pod \"e77e3016-6c58-4fac-82ca-589a93dba0ad\" (UID: \"e77e3016-6c58-4fac-82ca-589a93dba0ad\") " Sep 10 22:31:30.398689 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.398659 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e77e3016-6c58-4fac-82ca-589a93dba0ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e77e3016-6c58-4fac-82ca-589a93dba0ad" (UID: "e77e3016-6c58-4fac-82ca-589a93dba0ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:31:30.400387 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.400360 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e77e3016-6c58-4fac-82ca-589a93dba0ad-kube-api-access-gm6v2" (OuterVolumeSpecName: "kube-api-access-gm6v2") pod "e77e3016-6c58-4fac-82ca-589a93dba0ad" (UID: "e77e3016-6c58-4fac-82ca-589a93dba0ad"). InnerVolumeSpecName "kube-api-access-gm6v2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:31:30.499787 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.499742 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm6v2\" (UniqueName: \"kubernetes.io/projected/e77e3016-6c58-4fac-82ca-589a93dba0ad-kube-api-access-gm6v2\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:31:30.499787 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.499783 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e77e3016-6c58-4fac-82ca-589a93dba0ad-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:31:30.611371 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.611340 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:30.642586 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.642551 2576 generic.go:356] "Generic (PLEG): container finished" podID="e77e3016-6c58-4fac-82ca-589a93dba0ad" containerID="08b3daf6ff04a097339015cc9a3456efef779d7490d4171562057efe0422d639" exitCode=0 Sep 10 22:31:30.642779 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.642600 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" Sep 10 22:31:30.642779 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.642639 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" event={"ID":"e77e3016-6c58-4fac-82ca-589a93dba0ad","Type":"ContainerDied","Data":"08b3daf6ff04a097339015cc9a3456efef779d7490d4171562057efe0422d639"} Sep 10 22:31:30.642779 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.642696 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn" event={"ID":"e77e3016-6c58-4fac-82ca-589a93dba0ad","Type":"ContainerDied","Data":"26efc34d27e56ffecf9230b351fcb25c3d917801109141a2f2f0e314a992756b"} Sep 10 22:31:30.642779 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.642725 2576 scope.go:117] "RemoveContainer" containerID="08b3daf6ff04a097339015cc9a3456efef779d7490d4171562057efe0422d639" Sep 10 22:31:30.651989 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.651966 2576 scope.go:117] "RemoveContainer" containerID="08b3daf6ff04a097339015cc9a3456efef779d7490d4171562057efe0422d639" Sep 10 22:31:30.652313 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:31:30.652293 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08b3daf6ff04a097339015cc9a3456efef779d7490d4171562057efe0422d639\": container with ID starting with 08b3daf6ff04a097339015cc9a3456efef779d7490d4171562057efe0422d639 not found: ID does not exist" containerID="08b3daf6ff04a097339015cc9a3456efef779d7490d4171562057efe0422d639" Sep 10 22:31:30.652369 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.652325 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08b3daf6ff04a097339015cc9a3456efef779d7490d4171562057efe0422d639"} err="failed to get container status \"08b3daf6ff04a097339015cc9a3456efef779d7490d4171562057efe0422d639\": rpc error: code = NotFound desc = could not find container \"08b3daf6ff04a097339015cc9a3456efef779d7490d4171562057efe0422d639\": container with ID starting with 08b3daf6ff04a097339015cc9a3456efef779d7490d4171562057efe0422d639 not found: ID does not exist" Sep 10 22:31:30.660150 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.660120 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn"] Sep 10 22:31:30.663892 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:30.663862 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-79434a-98c54f6f5-vxrcn"] Sep 10 22:31:32.431911 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:32.431877 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e77e3016-6c58-4fac-82ca-589a93dba0ad" path="/var/lib/kubelet/pods/e77e3016-6c58-4fac-82ca-589a93dba0ad/volumes" Sep 10 22:31:34.891854 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:34.891820 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:37.083302 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:37.083269 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:37.158669 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:37.158636 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:37.359436 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:37.359348 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:37.823599 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:37.823567 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:48.259132 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.259092 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf"] Sep 10 22:31:48.260206 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.260008 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" podUID="5e0a2a8f-1f81-4674-b8f9-dd2997485c8a" containerName="maas-api" containerID="cri-o://c922636493540ac1b35b7526c5c9ce78af561d5238af42dd35fa7ae84525cc14" gracePeriod=30 Sep 10 22:31:48.292901 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.292861 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d07dfb-29817990-tgb59"] Sep 10 22:31:48.296052 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.296017 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d07dfb-29817990-tgb59"] Sep 10 22:31:48.296888 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.296864 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:48.330776 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.330743 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw"] Sep 10 22:31:48.330991 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.330967 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" podUID="da7b2577-b37e-40f3-be88-047fb925454b" containerName="payload-pre-processing" containerID="cri-o://5a3ca69641eab9038ac0e6dcc70dae2dd0c46619679ab95f36d990e10bc4a929" gracePeriod=30 Sep 10 22:31:48.371716 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.371605 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:48.432236 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.432203 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8a36382-4699-4d59-8d6b-b392fea43829" path="/var/lib/kubelet/pods/b8a36382-4699-4d59-8d6b-b392fea43829/volumes" Sep 10 22:31:48.472646 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.472569 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:48.573090 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.573051 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:48.587597 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.587572 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:31:48.617833 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.617786 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" Sep 10 22:31:48.686418 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.686378 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-openshift-service-ca\") pod \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " Sep 10 22:31:48.686418 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.686422 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-maas-api-tls\") pod \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " Sep 10 22:31:48.686670 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.686482 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6266\" (UniqueName: \"kubernetes.io/projected/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-kube-api-access-x6266\") pod \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " Sep 10 22:31:48.686670 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.686526 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-metrics-certs\") pod \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\" (UID: \"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a\") " Sep 10 22:31:48.686849 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.686817 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5e0a2a8f-1f81-4674-b8f9-dd2997485c8a" (UID: "5e0a2a8f-1f81-4674-b8f9-dd2997485c8a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:31:48.688663 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.688627 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-kube-api-access-x6266" (OuterVolumeSpecName: "kube-api-access-x6266") pod "5e0a2a8f-1f81-4674-b8f9-dd2997485c8a" (UID: "5e0a2a8f-1f81-4674-b8f9-dd2997485c8a"). InnerVolumeSpecName "kube-api-access-x6266". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:31:48.688663 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.688646 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5e0a2a8f-1f81-4674-b8f9-dd2997485c8a" (UID: "5e0a2a8f-1f81-4674-b8f9-dd2997485c8a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:31:48.688663 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.688664 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5e0a2a8f-1f81-4674-b8f9-dd2997485c8a" (UID: "5e0a2a8f-1f81-4674-b8f9-dd2997485c8a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:31:48.741177 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.741144 2576 generic.go:356] "Generic (PLEG): container finished" podID="da7b2577-b37e-40f3-be88-047fb925454b" containerID="5a3ca69641eab9038ac0e6dcc70dae2dd0c46619679ab95f36d990e10bc4a929" exitCode=0 Sep 10 22:31:48.741409 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.741192 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" Sep 10 22:31:48.741409 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.741235 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" event={"ID":"da7b2577-b37e-40f3-be88-047fb925454b","Type":"ContainerDied","Data":"5a3ca69641eab9038ac0e6dcc70dae2dd0c46619679ab95f36d990e10bc4a929"} Sep 10 22:31:48.741409 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.741300 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw" event={"ID":"da7b2577-b37e-40f3-be88-047fb925454b","Type":"ContainerDied","Data":"d84f5c6f8406fa9e6b07a235874818eea2595d3a1c5880eb49975b28d6497070"} Sep 10 22:31:48.741409 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.741331 2576 scope.go:117] "RemoveContainer" containerID="5a3ca69641eab9038ac0e6dcc70dae2dd0c46619679ab95f36d990e10bc4a929" Sep 10 22:31:48.742766 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.742744 2576 generic.go:356] "Generic (PLEG): container finished" podID="5e0a2a8f-1f81-4674-b8f9-dd2997485c8a" containerID="c922636493540ac1b35b7526c5c9ce78af561d5238af42dd35fa7ae84525cc14" exitCode=0 Sep 10 22:31:48.742891 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.742780 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" event={"ID":"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a","Type":"ContainerDied","Data":"c922636493540ac1b35b7526c5c9ce78af561d5238af42dd35fa7ae84525cc14"} Sep 10 22:31:48.742891 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.742795 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" Sep 10 22:31:48.742891 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.742849 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf" event={"ID":"5e0a2a8f-1f81-4674-b8f9-dd2997485c8a","Type":"ContainerDied","Data":"97e2710391e53424df3421f14bd29ffc597da41b564892efd419259ffb14c2f5"} Sep 10 22:31:48.753426 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.753238 2576 scope.go:117] "RemoveContainer" containerID="5a3ca69641eab9038ac0e6dcc70dae2dd0c46619679ab95f36d990e10bc4a929" Sep 10 22:31:48.755226 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:31:48.755195 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a3ca69641eab9038ac0e6dcc70dae2dd0c46619679ab95f36d990e10bc4a929\": container with ID starting with 5a3ca69641eab9038ac0e6dcc70dae2dd0c46619679ab95f36d990e10bc4a929 not found: ID does not exist" containerID="5a3ca69641eab9038ac0e6dcc70dae2dd0c46619679ab95f36d990e10bc4a929" Sep 10 22:31:48.755359 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.755232 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a3ca69641eab9038ac0e6dcc70dae2dd0c46619679ab95f36d990e10bc4a929"} err="failed to get container status \"5a3ca69641eab9038ac0e6dcc70dae2dd0c46619679ab95f36d990e10bc4a929\": rpc error: code = NotFound desc = could not find container \"5a3ca69641eab9038ac0e6dcc70dae2dd0c46619679ab95f36d990e10bc4a929\": container with ID starting with 5a3ca69641eab9038ac0e6dcc70dae2dd0c46619679ab95f36d990e10bc4a929 not found: ID does not exist" Sep 10 22:31:48.755359 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.755253 2576 scope.go:117] "RemoveContainer" containerID="c922636493540ac1b35b7526c5c9ce78af561d5238af42dd35fa7ae84525cc14" Sep 10 22:31:48.764446 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.764420 2576 scope.go:117] "RemoveContainer" containerID="c922636493540ac1b35b7526c5c9ce78af561d5238af42dd35fa7ae84525cc14" Sep 10 22:31:48.764780 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:31:48.764759 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c922636493540ac1b35b7526c5c9ce78af561d5238af42dd35fa7ae84525cc14\": container with ID starting with c922636493540ac1b35b7526c5c9ce78af561d5238af42dd35fa7ae84525cc14 not found: ID does not exist" containerID="c922636493540ac1b35b7526c5c9ce78af561d5238af42dd35fa7ae84525cc14" Sep 10 22:31:48.764917 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.764790 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c922636493540ac1b35b7526c5c9ce78af561d5238af42dd35fa7ae84525cc14"} err="failed to get container status \"c922636493540ac1b35b7526c5c9ce78af561d5238af42dd35fa7ae84525cc14\": rpc error: code = NotFound desc = could not find container \"c922636493540ac1b35b7526c5c9ce78af561d5238af42dd35fa7ae84525cc14\": container with ID starting with c922636493540ac1b35b7526c5c9ce78af561d5238af42dd35fa7ae84525cc14 not found: ID does not exist" Sep 10 22:31:48.767541 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.767513 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf"] Sep 10 22:31:48.771086 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.771051 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d07dfb-cd565b7b5-blkmf"] Sep 10 22:31:48.787191 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.787099 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vv4wq\" (UniqueName: \"kubernetes.io/projected/da7b2577-b37e-40f3-be88-047fb925454b-kube-api-access-vv4wq\") pod \"da7b2577-b37e-40f3-be88-047fb925454b\" (UID: \"da7b2577-b37e-40f3-be88-047fb925454b\") " Sep 10 22:31:48.787191 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.787150 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da7b2577-b37e-40f3-be88-047fb925454b-plugins-config-volume\") pod \"da7b2577-b37e-40f3-be88-047fb925454b\" (UID: \"da7b2577-b37e-40f3-be88-047fb925454b\") " Sep 10 22:31:48.787518 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.787502 2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-metrics-certs\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:31:48.787604 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.787524 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-openshift-service-ca\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:31:48.787604 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.787539 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-maas-api-tls\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:31:48.787604 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.787553 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6266\" (UniqueName: \"kubernetes.io/projected/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a-kube-api-access-x6266\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:31:48.787604 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.787548 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da7b2577-b37e-40f3-be88-047fb925454b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "da7b2577-b37e-40f3-be88-047fb925454b" (UID: "da7b2577-b37e-40f3-be88-047fb925454b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:31:48.789323 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.789293 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da7b2577-b37e-40f3-be88-047fb925454b-kube-api-access-vv4wq" (OuterVolumeSpecName: "kube-api-access-vv4wq") pod "da7b2577-b37e-40f3-be88-047fb925454b" (UID: "da7b2577-b37e-40f3-be88-047fb925454b"). InnerVolumeSpecName "kube-api-access-vv4wq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:31:48.888244 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.888200 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv4wq\" (UniqueName: \"kubernetes.io/projected/da7b2577-b37e-40f3-be88-047fb925454b-kube-api-access-vv4wq\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:31:48.888244 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.888234 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da7b2577-b37e-40f3-be88-047fb925454b-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:31:48.888844 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:48.888793 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:49.063542 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:49.063505 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw"] Sep 10 22:31:49.067893 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:49.067861 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d07dfb-9cc646db6-rcdsw"] Sep 10 22:31:50.433617 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:50.433572 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e0a2a8f-1f81-4674-b8f9-dd2997485c8a" path="/var/lib/kubelet/pods/5e0a2a8f-1f81-4674-b8f9-dd2997485c8a/volumes" Sep 10 22:31:50.434206 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:50.434184 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da7b2577-b37e-40f3-be88-047fb925454b" path="/var/lib/kubelet/pods/da7b2577-b37e-40f3-be88-047fb925454b/volumes" Sep 10 22:31:54.846996 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:54.846943 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:54.917309 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:54.917279 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:55.142834 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:55.142685 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:31:55.678813 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:31:55.678774 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:32:06.028666 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.028626 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:32:06.061007 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.060974 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762"] Sep 10 22:32:06.061230 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.061180 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" podUID="c98cb902-7d04-4795-a410-97b27db8a0ce" containerName="payload-processing" containerID="cri-o://7468aa0358144eeecff92920a43c84f84c6e0a6e3cf203ae8ad1b6f750e9fec8" gracePeriod=30 Sep 10 22:32:06.125526 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.125494 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:32:06.227281 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.227245 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:32:06.324462 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.324432 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:32:06.326711 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.326687 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" Sep 10 22:32:06.462964 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.462931 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c98cb902-7d04-4795-a410-97b27db8a0ce-plugins-config-volume\") pod \"c98cb902-7d04-4795-a410-97b27db8a0ce\" (UID: \"c98cb902-7d04-4795-a410-97b27db8a0ce\") " Sep 10 22:32:06.463146 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.463035 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ltxp\" (UniqueName: \"kubernetes.io/projected/c98cb902-7d04-4795-a410-97b27db8a0ce-kube-api-access-5ltxp\") pod \"c98cb902-7d04-4795-a410-97b27db8a0ce\" (UID: \"c98cb902-7d04-4795-a410-97b27db8a0ce\") " Sep 10 22:32:06.463308 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.463286 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c98cb902-7d04-4795-a410-97b27db8a0ce-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c98cb902-7d04-4795-a410-97b27db8a0ce" (UID: "c98cb902-7d04-4795-a410-97b27db8a0ce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:32:06.465200 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.465178 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c98cb902-7d04-4795-a410-97b27db8a0ce-kube-api-access-5ltxp" (OuterVolumeSpecName: "kube-api-access-5ltxp") pod "c98cb902-7d04-4795-a410-97b27db8a0ce" (UID: "c98cb902-7d04-4795-a410-97b27db8a0ce"). InnerVolumeSpecName "kube-api-access-5ltxp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:32:06.564625 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.564538 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ltxp\" (UniqueName: \"kubernetes.io/projected/c98cb902-7d04-4795-a410-97b27db8a0ce-kube-api-access-5ltxp\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:32:06.564625 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.564571 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c98cb902-7d04-4795-a410-97b27db8a0ce-plugins-config-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:32:06.667198 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.667132 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:32:06.841019 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.840985 2576 generic.go:356] "Generic (PLEG): container finished" podID="c98cb902-7d04-4795-a410-97b27db8a0ce" containerID="7468aa0358144eeecff92920a43c84f84c6e0a6e3cf203ae8ad1b6f750e9fec8" exitCode=0 Sep 10 22:32:06.841179 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.841037 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" Sep 10 22:32:06.841179 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.841076 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" event={"ID":"c98cb902-7d04-4795-a410-97b27db8a0ce","Type":"ContainerDied","Data":"7468aa0358144eeecff92920a43c84f84c6e0a6e3cf203ae8ad1b6f750e9fec8"} Sep 10 22:32:06.841179 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.841133 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762" event={"ID":"c98cb902-7d04-4795-a410-97b27db8a0ce","Type":"ContainerDied","Data":"dfcef012bd2c72fd3dfdd8ee0c4eb34dd95ca642218ae3f86a9821c6aa330f5d"} Sep 10 22:32:06.841179 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.841158 2576 scope.go:117] "RemoveContainer" containerID="7468aa0358144eeecff92920a43c84f84c6e0a6e3cf203ae8ad1b6f750e9fec8" Sep 10 22:32:06.853544 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.853510 2576 scope.go:117] "RemoveContainer" containerID="7468aa0358144eeecff92920a43c84f84c6e0a6e3cf203ae8ad1b6f750e9fec8" Sep 10 22:32:06.854115 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:32:06.854091 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7468aa0358144eeecff92920a43c84f84c6e0a6e3cf203ae8ad1b6f750e9fec8\": container with ID starting with 7468aa0358144eeecff92920a43c84f84c6e0a6e3cf203ae8ad1b6f750e9fec8 not found: ID does not exist" containerID="7468aa0358144eeecff92920a43c84f84c6e0a6e3cf203ae8ad1b6f750e9fec8" Sep 10 22:32:06.854250 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.854128 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7468aa0358144eeecff92920a43c84f84c6e0a6e3cf203ae8ad1b6f750e9fec8"} err="failed to get container status \"7468aa0358144eeecff92920a43c84f84c6e0a6e3cf203ae8ad1b6f750e9fec8\": rpc error: code = NotFound desc = could not find container \"7468aa0358144eeecff92920a43c84f84c6e0a6e3cf203ae8ad1b6f750e9fec8\": container with ID starting with 7468aa0358144eeecff92920a43c84f84c6e0a6e3cf203ae8ad1b6f750e9fec8 not found: ID does not exist" Sep 10 22:32:06.864089 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.864056 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762"] Sep 10 22:32:06.871752 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:06.871724 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ec6d47-6767f7c5c5-ls762"] Sep 10 22:32:08.437928 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:08.437892 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c98cb902-7d04-4795-a410-97b27db8a0ce" path="/var/lib/kubelet/pods/c98cb902-7d04-4795-a410-97b27db8a0ce/volumes" Sep 10 22:32:13.152177 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:13.152143 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:32:13.252930 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:13.252895 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:32:13.434046 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:13.433966 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:32:15.622355 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:15.622322 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:32:15.854558 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:15.854520 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85457bf75f-p8grg"] Sep 10 22:32:15.854816 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:15.854728 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-85457bf75f-p8grg" podUID="58b36abc-694b-41f4-8f3a-a7d1263fa6f2" containerName="manager" containerID="cri-o://91e71e51f7a9e3fa688e0a7ef4032bf54b0ba0786d0086453eea206f81241a75" gracePeriod=30 Sep 10 22:32:16.111998 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.111966 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:32:16.262416 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.262321 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-webhook-cert\") pod \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\" (UID: \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\") " Sep 10 22:32:16.262416 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.262392 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdzlw\" (UniqueName: \"kubernetes.io/projected/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-kube-api-access-bdzlw\") pod \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\" (UID: \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\") " Sep 10 22:32:16.262648 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.262449 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-metrics-certs\") pod \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\" (UID: \"58b36abc-694b-41f4-8f3a-a7d1263fa6f2\") " Sep 10 22:32:16.264619 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.264582 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "58b36abc-694b-41f4-8f3a-a7d1263fa6f2" (UID: "58b36abc-694b-41f4-8f3a-a7d1263fa6f2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:32:16.264752 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.264638 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-kube-api-access-bdzlw" (OuterVolumeSpecName: "kube-api-access-bdzlw") pod "58b36abc-694b-41f4-8f3a-a7d1263fa6f2" (UID: "58b36abc-694b-41f4-8f3a-a7d1263fa6f2"). InnerVolumeSpecName "kube-api-access-bdzlw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:32:16.264752 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.264640 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "58b36abc-694b-41f4-8f3a-a7d1263fa6f2" (UID: "58b36abc-694b-41f4-8f3a-a7d1263fa6f2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:32:16.363602 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.363560 2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-webhook-cert\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:32:16.363602 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.363594 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdzlw\" (UniqueName: \"kubernetes.io/projected/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-kube-api-access-bdzlw\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:32:16.363602 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.363603 2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58b36abc-694b-41f4-8f3a-a7d1263fa6f2-metrics-certs\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:32:16.894439 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.894401 2576 generic.go:356] "Generic (PLEG): container finished" podID="58b36abc-694b-41f4-8f3a-a7d1263fa6f2" containerID="91e71e51f7a9e3fa688e0a7ef4032bf54b0ba0786d0086453eea206f81241a75" exitCode=0 Sep 10 22:32:16.894877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.894447 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85457bf75f-p8grg" Sep 10 22:32:16.894877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.894488 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85457bf75f-p8grg" event={"ID":"58b36abc-694b-41f4-8f3a-a7d1263fa6f2","Type":"ContainerDied","Data":"91e71e51f7a9e3fa688e0a7ef4032bf54b0ba0786d0086453eea206f81241a75"} Sep 10 22:32:16.894877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.894544 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85457bf75f-p8grg" event={"ID":"58b36abc-694b-41f4-8f3a-a7d1263fa6f2","Type":"ContainerDied","Data":"8882513a9c7b6b7ccb56895bf1b0cdcff5020afe20436938b4d6bb982ae52168"} Sep 10 22:32:16.894877 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.894569 2576 scope.go:117] "RemoveContainer" containerID="91e71e51f7a9e3fa688e0a7ef4032bf54b0ba0786d0086453eea206f81241a75" Sep 10 22:32:16.904357 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.904335 2576 scope.go:117] "RemoveContainer" containerID="91e71e51f7a9e3fa688e0a7ef4032bf54b0ba0786d0086453eea206f81241a75" Sep 10 22:32:16.904691 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:32:16.904669 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91e71e51f7a9e3fa688e0a7ef4032bf54b0ba0786d0086453eea206f81241a75\": container with ID starting with 91e71e51f7a9e3fa688e0a7ef4032bf54b0ba0786d0086453eea206f81241a75 not found: ID does not exist" containerID="91e71e51f7a9e3fa688e0a7ef4032bf54b0ba0786d0086453eea206f81241a75" Sep 10 22:32:16.904773 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.904699 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91e71e51f7a9e3fa688e0a7ef4032bf54b0ba0786d0086453eea206f81241a75"} err="failed to get container status \"91e71e51f7a9e3fa688e0a7ef4032bf54b0ba0786d0086453eea206f81241a75\": rpc error: code = NotFound desc = could not find container \"91e71e51f7a9e3fa688e0a7ef4032bf54b0ba0786d0086453eea206f81241a75\": container with ID starting with 91e71e51f7a9e3fa688e0a7ef4032bf54b0ba0786d0086453eea206f81241a75 not found: ID does not exist" Sep 10 22:32:16.913127 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.913092 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85457bf75f-p8grg"] Sep 10 22:32:16.922161 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:16.922132 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-85457bf75f-p8grg"] Sep 10 22:32:18.431952 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:32:18.431918 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58b36abc-694b-41f4-8f3a-a7d1263fa6f2" path="/var/lib/kubelet/pods/58b36abc-694b-41f4-8f3a-a7d1263fa6f2/volumes" Sep 10 22:33:17.728625 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.728586 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-85457bf75f-c8c95"] Sep 10 22:33:17.729064 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729043 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da7b2577-b37e-40f3-be88-047fb925454b" containerName="payload-pre-processing" Sep 10 22:33:17.729064 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729056 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="da7b2577-b37e-40f3-be88-047fb925454b" containerName="payload-pre-processing" Sep 10 22:33:17.729064 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729066 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58b36abc-694b-41f4-8f3a-a7d1263fa6f2" containerName="manager" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729072 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="58b36abc-694b-41f4-8f3a-a7d1263fa6f2" containerName="manager" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729079 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c98cb902-7d04-4795-a410-97b27db8a0ce" containerName="payload-processing" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729085 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c98cb902-7d04-4795-a410-97b27db8a0ce" containerName="payload-processing" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729093 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0e082a2-5efd-43af-b111-bdaf33e210e3" containerName="payload-pre-processing" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729099 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0e082a2-5efd-43af-b111-bdaf33e210e3" containerName="payload-pre-processing" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729105 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0b1baa2-6e96-4c02-a4fd-0ec5718995ef" containerName="payload-processing" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729111 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0b1baa2-6e96-4c02-a4fd-0ec5718995ef" containerName="payload-processing" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729117 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24c48036-3df4-4851-9819-308f0a33abb1" containerName="cleanup" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729122 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="24c48036-3df4-4851-9819-308f0a33abb1" containerName="cleanup" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729134 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8a36382-4699-4d59-8d6b-b392fea43829" containerName="cleanup" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729140 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8a36382-4699-4d59-8d6b-b392fea43829" containerName="cleanup" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729148 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e77e3016-6c58-4fac-82ca-589a93dba0ad" containerName="payload-processing" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729153 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e77e3016-6c58-4fac-82ca-589a93dba0ad" containerName="payload-processing" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729160 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e0a2a8f-1f81-4674-b8f9-dd2997485c8a" containerName="maas-api" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729165 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e0a2a8f-1f81-4674-b8f9-dd2997485c8a" containerName="maas-api" Sep 10 22:33:17.729167 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729171 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a49e215a-af8d-43b5-b9cb-2bf8ab232da3" containerName="payload-processing" Sep 10 22:33:17.729643 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729176 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a49e215a-af8d-43b5-b9cb-2bf8ab232da3" containerName="payload-processing" Sep 10 22:33:17.729643 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729229 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8a36382-4699-4d59-8d6b-b392fea43829" containerName="cleanup" Sep 10 22:33:17.729643 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729236 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c98cb902-7d04-4795-a410-97b27db8a0ce" containerName="payload-processing" Sep 10 22:33:17.729643 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729242 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a49e215a-af8d-43b5-b9cb-2bf8ab232da3" containerName="payload-processing" Sep 10 22:33:17.729643 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729251 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="da7b2577-b37e-40f3-be88-047fb925454b" containerName="payload-pre-processing" Sep 10 22:33:17.729643 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729257 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e0a2a8f-1f81-4674-b8f9-dd2997485c8a" containerName="maas-api" Sep 10 22:33:17.729643 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729263 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0b1baa2-6e96-4c02-a4fd-0ec5718995ef" containerName="payload-processing" Sep 10 22:33:17.729643 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729269 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="58b36abc-694b-41f4-8f3a-a7d1263fa6f2" containerName="manager" Sep 10 22:33:17.729643 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729274 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e77e3016-6c58-4fac-82ca-589a93dba0ad" containerName="payload-processing" Sep 10 22:33:17.729643 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729280 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="24c48036-3df4-4851-9819-308f0a33abb1" containerName="cleanup" Sep 10 22:33:17.729643 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.729287 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0e082a2-5efd-43af-b111-bdaf33e210e3" containerName="payload-pre-processing" Sep 10 22:33:17.732399 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.732380 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85457bf75f-c8c95" Sep 10 22:33:17.735355 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.735329 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 10 22:33:17.735513 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.735334 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 10 22:33:17.735513 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.735335 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 10 22:33:17.735513 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.735462 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5h96d\"" Sep 10 22:33:17.742498 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.742423 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85457bf75f-c8c95"] Sep 10 22:33:17.865902 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.865857 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60a8428b-3670-434a-a965-48adf66d8730-webhook-cert\") pod \"maas-controller-85457bf75f-c8c95\" (UID: \"60a8428b-3670-434a-a965-48adf66d8730\") " pod="opendatahub/maas-controller-85457bf75f-c8c95" Sep 10 22:33:17.866114 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.866018 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60a8428b-3670-434a-a965-48adf66d8730-metrics-certs\") pod \"maas-controller-85457bf75f-c8c95\" (UID: \"60a8428b-3670-434a-a965-48adf66d8730\") " pod="opendatahub/maas-controller-85457bf75f-c8c95" Sep 10 22:33:17.866114 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.866060 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzlkv\" (UniqueName: \"kubernetes.io/projected/60a8428b-3670-434a-a965-48adf66d8730-kube-api-access-wzlkv\") pod \"maas-controller-85457bf75f-c8c95\" (UID: \"60a8428b-3670-434a-a965-48adf66d8730\") " pod="opendatahub/maas-controller-85457bf75f-c8c95" Sep 10 22:33:17.967096 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.967056 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60a8428b-3670-434a-a965-48adf66d8730-metrics-certs\") pod \"maas-controller-85457bf75f-c8c95\" (UID: \"60a8428b-3670-434a-a965-48adf66d8730\") " pod="opendatahub/maas-controller-85457bf75f-c8c95" Sep 10 22:33:17.967096 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.967095 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzlkv\" (UniqueName: \"kubernetes.io/projected/60a8428b-3670-434a-a965-48adf66d8730-kube-api-access-wzlkv\") pod \"maas-controller-85457bf75f-c8c95\" (UID: \"60a8428b-3670-434a-a965-48adf66d8730\") " pod="opendatahub/maas-controller-85457bf75f-c8c95" Sep 10 22:33:17.967374 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.967148 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60a8428b-3670-434a-a965-48adf66d8730-webhook-cert\") pod \"maas-controller-85457bf75f-c8c95\" (UID: \"60a8428b-3670-434a-a965-48adf66d8730\") " pod="opendatahub/maas-controller-85457bf75f-c8c95" Sep 10 22:33:17.969711 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.969677 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60a8428b-3670-434a-a965-48adf66d8730-metrics-certs\") pod \"maas-controller-85457bf75f-c8c95\" (UID: \"60a8428b-3670-434a-a965-48adf66d8730\") " pod="opendatahub/maas-controller-85457bf75f-c8c95" Sep 10 22:33:17.969859 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.969724 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60a8428b-3670-434a-a965-48adf66d8730-webhook-cert\") pod \"maas-controller-85457bf75f-c8c95\" (UID: \"60a8428b-3670-434a-a965-48adf66d8730\") " pod="opendatahub/maas-controller-85457bf75f-c8c95" Sep 10 22:33:17.976190 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:17.976163 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzlkv\" (UniqueName: \"kubernetes.io/projected/60a8428b-3670-434a-a965-48adf66d8730-kube-api-access-wzlkv\") pod \"maas-controller-85457bf75f-c8c95\" (UID: \"60a8428b-3670-434a-a965-48adf66d8730\") " pod="opendatahub/maas-controller-85457bf75f-c8c95" Sep 10 22:33:18.045080 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:18.044986 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85457bf75f-c8c95" Sep 10 22:33:18.321033 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:18.321007 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85457bf75f-c8c95"] Sep 10 22:33:18.322885 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:33:18.322852 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60a8428b_3670_434a_a965_48adf66d8730.slice/crio-01593282417adfd2609be009b9584a8bace1d0fbeea50cbd300dfa2588d0a968 WatchSource:0}: Error finding container 01593282417adfd2609be009b9584a8bace1d0fbeea50cbd300dfa2588d0a968: Status 404 returned error can't find the container with id 01593282417adfd2609be009b9584a8bace1d0fbeea50cbd300dfa2588d0a968 Sep 10 22:33:19.203729 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:19.203626 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85457bf75f-c8c95" event={"ID":"60a8428b-3670-434a-a965-48adf66d8730","Type":"ContainerStarted","Data":"b2bd4040b52fea569dbb5b43a3c6bae9608bfec32ea5c237ecbd6b98b2d82e81"} Sep 10 22:33:19.203729 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:19.203682 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85457bf75f-c8c95" event={"ID":"60a8428b-3670-434a-a965-48adf66d8730","Type":"ContainerStarted","Data":"01593282417adfd2609be009b9584a8bace1d0fbeea50cbd300dfa2588d0a968"} Sep 10 22:33:19.204172 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:19.203815 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-85457bf75f-c8c95" Sep 10 22:33:19.224835 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:19.224759 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-85457bf75f-c8c95" podStartSLOduration=1.727568208 podStartE2EDuration="2.224741151s" totalImagesPullingTime="497.172943ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:33:19.223092661 +0000 UTC m=+1929.344791453" watchObservedRunningTime="2026-09-10 22:33:19.224741151 +0000 UTC m=+1929.346439933" Sep 10 22:33:30.211309 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:30.211272 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-85457bf75f-c8c95" Sep 10 22:33:41.597028 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:41.596990 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:33:41.713370 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:41.713327 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85457bf75f-c8c95"] Sep 10 22:33:41.713678 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:41.713597 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-85457bf75f-c8c95" podUID="60a8428b-3670-434a-a965-48adf66d8730" containerName="manager" containerID="cri-o://b2bd4040b52fea569dbb5b43a3c6bae9608bfec32ea5c237ecbd6b98b2d82e81" gracePeriod=30 Sep 10 22:33:41.973430 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:41.973399 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85457bf75f-c8c95" Sep 10 22:33:42.006638 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.006602 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60a8428b-3670-434a-a965-48adf66d8730-metrics-certs\") pod \"60a8428b-3670-434a-a965-48adf66d8730\" (UID: \"60a8428b-3670-434a-a965-48adf66d8730\") " Sep 10 22:33:42.006844 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.006690 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzlkv\" (UniqueName: \"kubernetes.io/projected/60a8428b-3670-434a-a965-48adf66d8730-kube-api-access-wzlkv\") pod \"60a8428b-3670-434a-a965-48adf66d8730\" (UID: \"60a8428b-3670-434a-a965-48adf66d8730\") " Sep 10 22:33:42.006844 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.006771 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60a8428b-3670-434a-a965-48adf66d8730-webhook-cert\") pod \"60a8428b-3670-434a-a965-48adf66d8730\" (UID: \"60a8428b-3670-434a-a965-48adf66d8730\") " Sep 10 22:33:42.009029 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.008926 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60a8428b-3670-434a-a965-48adf66d8730-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "60a8428b-3670-434a-a965-48adf66d8730" (UID: "60a8428b-3670-434a-a965-48adf66d8730"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:33:42.009400 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.009363 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60a8428b-3670-434a-a965-48adf66d8730-kube-api-access-wzlkv" (OuterVolumeSpecName: "kube-api-access-wzlkv") pod "60a8428b-3670-434a-a965-48adf66d8730" (UID: "60a8428b-3670-434a-a965-48adf66d8730"). InnerVolumeSpecName "kube-api-access-wzlkv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:33:42.009400 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.009370 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60a8428b-3670-434a-a965-48adf66d8730-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "60a8428b-3670-434a-a965-48adf66d8730" (UID: "60a8428b-3670-434a-a965-48adf66d8730"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:33:42.108370 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.108330 2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60a8428b-3670-434a-a965-48adf66d8730-metrics-certs\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:33:42.108370 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.108363 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzlkv\" (UniqueName: \"kubernetes.io/projected/60a8428b-3670-434a-a965-48adf66d8730-kube-api-access-wzlkv\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:33:42.108370 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.108373 2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60a8428b-3670-434a-a965-48adf66d8730-webhook-cert\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:33:42.318563 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.318530 2576 generic.go:356] "Generic (PLEG): container finished" podID="60a8428b-3670-434a-a965-48adf66d8730" containerID="b2bd4040b52fea569dbb5b43a3c6bae9608bfec32ea5c237ecbd6b98b2d82e81" exitCode=0 Sep 10 22:33:42.318744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.318580 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85457bf75f-c8c95" Sep 10 22:33:42.318744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.318616 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85457bf75f-c8c95" event={"ID":"60a8428b-3670-434a-a965-48adf66d8730","Type":"ContainerDied","Data":"b2bd4040b52fea569dbb5b43a3c6bae9608bfec32ea5c237ecbd6b98b2d82e81"} Sep 10 22:33:42.318744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.318680 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85457bf75f-c8c95" event={"ID":"60a8428b-3670-434a-a965-48adf66d8730","Type":"ContainerDied","Data":"01593282417adfd2609be009b9584a8bace1d0fbeea50cbd300dfa2588d0a968"} Sep 10 22:33:42.318744 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.318704 2576 scope.go:117] "RemoveContainer" containerID="b2bd4040b52fea569dbb5b43a3c6bae9608bfec32ea5c237ecbd6b98b2d82e81" Sep 10 22:33:42.328860 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.328840 2576 scope.go:117] "RemoveContainer" containerID="b2bd4040b52fea569dbb5b43a3c6bae9608bfec32ea5c237ecbd6b98b2d82e81" Sep 10 22:33:42.329198 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:33:42.329179 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2bd4040b52fea569dbb5b43a3c6bae9608bfec32ea5c237ecbd6b98b2d82e81\": container with ID starting with b2bd4040b52fea569dbb5b43a3c6bae9608bfec32ea5c237ecbd6b98b2d82e81 not found: ID does not exist" containerID="b2bd4040b52fea569dbb5b43a3c6bae9608bfec32ea5c237ecbd6b98b2d82e81" Sep 10 22:33:42.329272 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.329205 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2bd4040b52fea569dbb5b43a3c6bae9608bfec32ea5c237ecbd6b98b2d82e81"} err="failed to get container status \"b2bd4040b52fea569dbb5b43a3c6bae9608bfec32ea5c237ecbd6b98b2d82e81\": rpc error: code = NotFound desc = could not find container \"b2bd4040b52fea569dbb5b43a3c6bae9608bfec32ea5c237ecbd6b98b2d82e81\": container with ID starting with b2bd4040b52fea569dbb5b43a3c6bae9608bfec32ea5c237ecbd6b98b2d82e81 not found: ID does not exist" Sep 10 22:33:42.340691 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.340657 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85457bf75f-c8c95"] Sep 10 22:33:42.345066 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.345031 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-85457bf75f-c8c95"] Sep 10 22:33:42.432638 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:42.432603 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60a8428b-3670-434a-a965-48adf66d8730" path="/var/lib/kubelet/pods/60a8428b-3670-434a-a965-48adf66d8730/volumes" Sep 10 22:33:43.029422 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.029369 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:33:43.135202 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.135166 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-85457bf75f-kk854"] Sep 10 22:33:43.135584 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.135571 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60a8428b-3670-434a-a965-48adf66d8730" containerName="manager" Sep 10 22:33:43.135584 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.135585 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="60a8428b-3670-434a-a965-48adf66d8730" containerName="manager" Sep 10 22:33:43.135688 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.135641 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="60a8428b-3670-434a-a965-48adf66d8730" containerName="manager" Sep 10 22:33:43.140268 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.140243 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85457bf75f-kk854" Sep 10 22:33:43.143055 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.143026 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 10 22:33:43.143055 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.143053 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 10 22:33:43.143246 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.143054 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5h96d\"" Sep 10 22:33:43.143246 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.143124 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 10 22:33:43.150435 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.150405 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85457bf75f-kk854"] Sep 10 22:33:43.220133 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.220090 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e605682f-7800-4c7d-9715-c4f1edeed031-metrics-certs\") pod \"maas-controller-85457bf75f-kk854\" (UID: \"e605682f-7800-4c7d-9715-c4f1edeed031\") " pod="opendatahub/maas-controller-85457bf75f-kk854" Sep 10 22:33:43.220320 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.220191 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e605682f-7800-4c7d-9715-c4f1edeed031-webhook-cert\") pod \"maas-controller-85457bf75f-kk854\" (UID: \"e605682f-7800-4c7d-9715-c4f1edeed031\") " pod="opendatahub/maas-controller-85457bf75f-kk854" Sep 10 22:33:43.220320 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.220235 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpvqw\" (UniqueName: \"kubernetes.io/projected/e605682f-7800-4c7d-9715-c4f1edeed031-kube-api-access-jpvqw\") pod \"maas-controller-85457bf75f-kk854\" (UID: \"e605682f-7800-4c7d-9715-c4f1edeed031\") " pod="opendatahub/maas-controller-85457bf75f-kk854" Sep 10 22:33:43.321186 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.321161 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e605682f-7800-4c7d-9715-c4f1edeed031-webhook-cert\") pod \"maas-controller-85457bf75f-kk854\" (UID: \"e605682f-7800-4c7d-9715-c4f1edeed031\") " pod="opendatahub/maas-controller-85457bf75f-kk854" Sep 10 22:33:43.321375 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.321209 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpvqw\" (UniqueName: \"kubernetes.io/projected/e605682f-7800-4c7d-9715-c4f1edeed031-kube-api-access-jpvqw\") pod \"maas-controller-85457bf75f-kk854\" (UID: \"e605682f-7800-4c7d-9715-c4f1edeed031\") " pod="opendatahub/maas-controller-85457bf75f-kk854" Sep 10 22:33:43.321375 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.321254 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e605682f-7800-4c7d-9715-c4f1edeed031-metrics-certs\") pod \"maas-controller-85457bf75f-kk854\" (UID: \"e605682f-7800-4c7d-9715-c4f1edeed031\") " pod="opendatahub/maas-controller-85457bf75f-kk854" Sep 10 22:33:43.323851 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.323827 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e605682f-7800-4c7d-9715-c4f1edeed031-metrics-certs\") pod \"maas-controller-85457bf75f-kk854\" (UID: \"e605682f-7800-4c7d-9715-c4f1edeed031\") " pod="opendatahub/maas-controller-85457bf75f-kk854" Sep 10 22:33:43.323982 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.323881 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e605682f-7800-4c7d-9715-c4f1edeed031-webhook-cert\") pod \"maas-controller-85457bf75f-kk854\" (UID: \"e605682f-7800-4c7d-9715-c4f1edeed031\") " pod="opendatahub/maas-controller-85457bf75f-kk854" Sep 10 22:33:43.328563 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.328535 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpvqw\" (UniqueName: \"kubernetes.io/projected/e605682f-7800-4c7d-9715-c4f1edeed031-kube-api-access-jpvqw\") pod \"maas-controller-85457bf75f-kk854\" (UID: \"e605682f-7800-4c7d-9715-c4f1edeed031\") " pod="opendatahub/maas-controller-85457bf75f-kk854" Sep 10 22:33:43.452179 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.452137 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85457bf75f-kk854" Sep 10 22:33:43.728818 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:43.728717 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85457bf75f-kk854"] Sep 10 22:33:43.732392 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:33:43.732364 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode605682f_7800_4c7d_9715_c4f1edeed031.slice/crio-8bc75c02ba50957977e8b2ce8c34a148505304a7dd7918e5afc28bdbdc84f44e WatchSource:0}: Error finding container 8bc75c02ba50957977e8b2ce8c34a148505304a7dd7918e5afc28bdbdc84f44e: Status 404 returned error can't find the container with id 8bc75c02ba50957977e8b2ce8c34a148505304a7dd7918e5afc28bdbdc84f44e Sep 10 22:33:44.333534 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:44.333502 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85457bf75f-kk854" event={"ID":"e605682f-7800-4c7d-9715-c4f1edeed031","Type":"ContainerStarted","Data":"441526cf950748fc1149470ddd2ab421cbb521183bab82a51739775ee4289f14"} Sep 10 22:33:44.334015 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:44.333554 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85457bf75f-kk854" event={"ID":"e605682f-7800-4c7d-9715-c4f1edeed031","Type":"ContainerStarted","Data":"8bc75c02ba50957977e8b2ce8c34a148505304a7dd7918e5afc28bdbdc84f44e"} Sep 10 22:33:44.334015 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:44.333583 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-85457bf75f-kk854" Sep 10 22:33:44.352122 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:44.352067 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-85457bf75f-kk854" podStartSLOduration=0.923612746 podStartE2EDuration="1.352049571s" totalImagesPullingTime="428.436825ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:33:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:33:44.34983343 +0000 UTC m=+1954.471532212" watchObservedRunningTime="2026-09-10 22:33:44.352049571 +0000 UTC m=+1954.473748353" Sep 10 22:33:52.222355 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:52.222322 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:33:55.340856 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:33:55.340818 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-85457bf75f-kk854" Sep 10 22:34:10.676917 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:10.676831 2576 scope.go:117] "RemoveContainer" containerID="bb967a5dbcb2157a5595ee5f7f6f165407f14c507288929184cff200cdb9410f" Sep 10 22:34:54.864720 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:54.864690 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:34:56.422329 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:56.422292 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj"] Sep 10 22:34:56.422717 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:56.422551 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" podUID="77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d" containerName="manager" containerID="cri-o://db935b4a69e950d226875af708d0e48b364089a9a1211c8f57391aa274dde4f2" gracePeriod=10 Sep 10 22:34:58.872435 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:58.872405 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" Sep 10 22:34:58.960894 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:58.960762 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d-extensions-socket-volume\") pod \"77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d\" (UID: \"77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d\") " Sep 10 22:34:58.960894 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:58.960874 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qkxx\" (UniqueName: \"kubernetes.io/projected/77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d-kube-api-access-9qkxx\") pod \"77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d\" (UID: \"77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d\") " Sep 10 22:34:58.961141 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:58.961114 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d" (UID: "77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:34:58.961219 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:58.961201 2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d-extensions-socket-volume\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:34:58.963077 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:58.963056 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d-kube-api-access-9qkxx" (OuterVolumeSpecName: "kube-api-access-9qkxx") pod "77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d" (UID: "77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d"). InnerVolumeSpecName "kube-api-access-9qkxx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:34:59.062562 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:59.062518 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qkxx\" (UniqueName: \"kubernetes.io/projected/77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d-kube-api-access-9qkxx\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:34:59.735426 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:59.735392 2576 generic.go:356] "Generic (PLEG): container finished" podID="77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d" containerID="db935b4a69e950d226875af708d0e48b364089a9a1211c8f57391aa274dde4f2" exitCode=0 Sep 10 22:34:59.735605 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:59.735438 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" Sep 10 22:34:59.735605 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:59.735443 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" event={"ID":"77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d","Type":"ContainerDied","Data":"db935b4a69e950d226875af708d0e48b364089a9a1211c8f57391aa274dde4f2"} Sep 10 22:34:59.735605 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:59.735542 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj" event={"ID":"77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d","Type":"ContainerDied","Data":"c2542d3867f8a071e4d6ca1d92d94f00435029307adfe1f66afd17586ce5e4b7"} Sep 10 22:34:59.735605 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:59.735564 2576 scope.go:117] "RemoveContainer" containerID="db935b4a69e950d226875af708d0e48b364089a9a1211c8f57391aa274dde4f2" Sep 10 22:34:59.745696 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:59.745534 2576 scope.go:117] "RemoveContainer" containerID="db935b4a69e950d226875af708d0e48b364089a9a1211c8f57391aa274dde4f2" Sep 10 22:34:59.745936 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:34:59.745916 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db935b4a69e950d226875af708d0e48b364089a9a1211c8f57391aa274dde4f2\": container with ID starting with db935b4a69e950d226875af708d0e48b364089a9a1211c8f57391aa274dde4f2 not found: ID does not exist" containerID="db935b4a69e950d226875af708d0e48b364089a9a1211c8f57391aa274dde4f2" Sep 10 22:34:59.745983 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:59.745947 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db935b4a69e950d226875af708d0e48b364089a9a1211c8f57391aa274dde4f2"} err="failed to get container status \"db935b4a69e950d226875af708d0e48b364089a9a1211c8f57391aa274dde4f2\": rpc error: code = NotFound desc = could not find container \"db935b4a69e950d226875af708d0e48b364089a9a1211c8f57391aa274dde4f2\": container with ID starting with db935b4a69e950d226875af708d0e48b364089a9a1211c8f57391aa274dde4f2 not found: ID does not exist" Sep 10 22:34:59.767861 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:59.767822 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj"] Sep 10 22:34:59.771564 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:34:59.771533 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q62vj"] Sep 10 22:35:00.432278 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:00.432246 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d" path="/var/lib/kubelet/pods/77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d/volumes" Sep 10 22:35:48.675049 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:48.675011 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:35:57.149815 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.149760 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm"] Sep 10 22:35:57.150640 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.150614 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d" containerName="manager" Sep 10 22:35:57.150721 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.150643 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d" containerName="manager" Sep 10 22:35:57.153413 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.153385 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="77b5b1d8-c52c-4e6a-b99c-46ce7b53aa9d" containerName="manager" Sep 10 22:35:57.156376 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.156359 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm" Sep 10 22:35:57.159067 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.159043 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 10 22:35:57.159206 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.159134 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6bd9z\"" Sep 10 22:35:57.159427 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.159411 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 10 22:35:57.166687 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.166658 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm"] Sep 10 22:35:57.336722 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.336670 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvrhs\" (UniqueName: \"kubernetes.io/projected/33b10c4d-b3a8-4c04-84d3-b9cf8d640d10-kube-api-access-rvrhs\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4m5fm\" (UID: \"33b10c4d-b3a8-4c04-84d3-b9cf8d640d10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm" Sep 10 22:35:57.336953 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.336761 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33b10c4d-b3a8-4c04-84d3-b9cf8d640d10-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4m5fm\" (UID: \"33b10c4d-b3a8-4c04-84d3-b9cf8d640d10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm" Sep 10 22:35:57.437848 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.437744 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvrhs\" (UniqueName: \"kubernetes.io/projected/33b10c4d-b3a8-4c04-84d3-b9cf8d640d10-kube-api-access-rvrhs\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4m5fm\" (UID: \"33b10c4d-b3a8-4c04-84d3-b9cf8d640d10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm" Sep 10 22:35:57.438003 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.437853 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33b10c4d-b3a8-4c04-84d3-b9cf8d640d10-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4m5fm\" (UID: \"33b10c4d-b3a8-4c04-84d3-b9cf8d640d10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm" Sep 10 22:35:57.438228 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.438211 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33b10c4d-b3a8-4c04-84d3-b9cf8d640d10-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4m5fm\" (UID: \"33b10c4d-b3a8-4c04-84d3-b9cf8d640d10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm" Sep 10 22:35:57.450833 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.450782 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvrhs\" (UniqueName: \"kubernetes.io/projected/33b10c4d-b3a8-4c04-84d3-b9cf8d640d10-kube-api-access-rvrhs\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4m5fm\" (UID: \"33b10c4d-b3a8-4c04-84d3-b9cf8d640d10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm" Sep 10 22:35:57.468332 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.468295 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm" Sep 10 22:35:57.757164 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:57.757030 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm"] Sep 10 22:35:57.759611 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:35:57.759582 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33b10c4d_b3a8_4c04_84d3_b9cf8d640d10.slice/crio-47b9714d91995d80fa8486e1bbf78629032c1399969c5d08f9a792f3356331fe WatchSource:0}: Error finding container 47b9714d91995d80fa8486e1bbf78629032c1399969c5d08f9a792f3356331fe: Status 404 returned error can't find the container with id 47b9714d91995d80fa8486e1bbf78629032c1399969c5d08f9a792f3356331fe Sep 10 22:35:58.016979 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:58.016859 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm" event={"ID":"33b10c4d-b3a8-4c04-84d3-b9cf8d640d10","Type":"ContainerStarted","Data":"5cad01ba9613250fd62e0075f3998a23fde8c7f863db0e09c56c9da2679120e0"} Sep 10 22:35:58.016979 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:58.016939 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm" event={"ID":"33b10c4d-b3a8-4c04-84d3-b9cf8d640d10","Type":"ContainerStarted","Data":"47b9714d91995d80fa8486e1bbf78629032c1399969c5d08f9a792f3356331fe"} Sep 10 22:35:58.017219 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:58.017056 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm" Sep 10 22:35:58.048701 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:35:58.048633 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm" podStartSLOduration=1.048616648 podStartE2EDuration="1.048616648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:35:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:35:58.04713878 +0000 UTC m=+2088.168837587" watchObservedRunningTime="2026-09-10 22:35:58.048616648 +0000 UTC m=+2088.170315430" Sep 10 22:36:09.025166 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:36:09.025136 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4m5fm" Sep 10 22:36:10.715559 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:36:10.715519 2576 scope.go:117] "RemoveContainer" containerID="4cf90d2d9e386c6b6f81231a1e78208f945a637f7ae0ade1d4546eedfffda72b" Sep 10 22:36:52.394037 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:36:52.394002 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:37:10.120559 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:37:10.120478 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:37:45.324323 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:37:45.324280 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:37:46.275283 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:37:46.275252 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:38:06.545578 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.545533 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-559477f47-xclh8"] Sep 10 22:38:06.549232 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.549208 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:06.563832 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.563784 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-559477f47-xclh8"] Sep 10 22:38:06.675236 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.675201 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e-metrics-certs\") pod \"maas-api-559477f47-xclh8\" (UID: \"4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e\") " pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:06.675399 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.675255 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e-maas-api-tls\") pod \"maas-api-559477f47-xclh8\" (UID: \"4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e\") " pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:06.675399 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.675358 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e-openshift-service-ca\") pod \"maas-api-559477f47-xclh8\" (UID: \"4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e\") " pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:06.675490 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.675401 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-967qc\" (UniqueName: \"kubernetes.io/projected/4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e-kube-api-access-967qc\") pod \"maas-api-559477f47-xclh8\" (UID: \"4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e\") " pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:06.776298 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.776255 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e-metrics-certs\") pod \"maas-api-559477f47-xclh8\" (UID: \"4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e\") " pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:06.776502 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.776341 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e-maas-api-tls\") pod \"maas-api-559477f47-xclh8\" (UID: \"4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e\") " pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:06.776502 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.776430 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e-openshift-service-ca\") pod \"maas-api-559477f47-xclh8\" (UID: \"4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e\") " pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:06.776502 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.776480 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-967qc\" (UniqueName: \"kubernetes.io/projected/4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e-kube-api-access-967qc\") pod \"maas-api-559477f47-xclh8\" (UID: \"4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e\") " pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:06.777383 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.777358 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e-openshift-service-ca\") pod \"maas-api-559477f47-xclh8\" (UID: \"4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e\") " pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:06.779736 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.779700 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e-metrics-certs\") pod \"maas-api-559477f47-xclh8\" (UID: \"4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e\") " pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:06.780003 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.779979 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e-maas-api-tls\") pod \"maas-api-559477f47-xclh8\" (UID: \"4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e\") " pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:06.787708 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.787677 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-967qc\" (UniqueName: \"kubernetes.io/projected/4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e-kube-api-access-967qc\") pod \"maas-api-559477f47-xclh8\" (UID: \"4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e\") " pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:06.859661 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:06.859627 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:07.136108 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:07.136082 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-559477f47-xclh8"] Sep 10 22:38:07.138849 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:38:07.138814 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ecbe2d4_ea45_4e89_8d3e_27ec4807c37e.slice/crio-77584f262d7c6ef8dee5ebdc4ee5a525c8c667df22d9c7f71b015f3cb975501e WatchSource:0}: Error finding container 77584f262d7c6ef8dee5ebdc4ee5a525c8c667df22d9c7f71b015f3cb975501e: Status 404 returned error can't find the container with id 77584f262d7c6ef8dee5ebdc4ee5a525c8c667df22d9c7f71b015f3cb975501e Sep 10 22:38:07.140268 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:07.140245 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 22:38:07.666723 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:07.666689 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" event={"ID":"4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e","Type":"ContainerStarted","Data":"090c8fa9f9dbd73fa01ad27104e014dece7bb314d99ba50c20bb81fb475b6b40"} Sep 10 22:38:07.667193 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:07.666741 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" event={"ID":"4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e","Type":"ContainerStarted","Data":"77584f262d7c6ef8dee5ebdc4ee5a525c8c667df22d9c7f71b015f3cb975501e"} Sep 10 22:38:07.667193 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:07.666773 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:07.686752 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:07.686703 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" podStartSLOduration=1.298058961 podStartE2EDuration="1.686673821s" totalImagesPullingTime="388.61486ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:38:07.684079396 +0000 UTC m=+2217.805778178" watchObservedRunningTime="2026-09-10 22:38:07.686673821 +0000 UTC m=+2217.808372602" Sep 10 22:38:13.677104 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:13.677074 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-559477f47-xclh8" Sep 10 22:38:21.119266 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:21.119224 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq"] Sep 10 22:38:21.119881 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:21.119522 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" podUID="4562dca2-c775-43a3-b449-3344e2acaf7d" containerName="maas-api" containerID="cri-o://a0bb9bf4a2c64ea5c16b4207f1a82cf77ca617667f22f015ff2a9600ec4f8567" gracePeriod=30 Sep 10 22:38:22.375055 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.375027 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:38:22.431186 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.431155 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4562dca2-c775-43a3-b449-3344e2acaf7d-metrics-certs\") pod \"4562dca2-c775-43a3-b449-3344e2acaf7d\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " Sep 10 22:38:22.431382 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.431205 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljvqc\" (UniqueName: \"kubernetes.io/projected/4562dca2-c775-43a3-b449-3344e2acaf7d-kube-api-access-ljvqc\") pod \"4562dca2-c775-43a3-b449-3344e2acaf7d\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " Sep 10 22:38:22.431382 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.431276 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4562dca2-c775-43a3-b449-3344e2acaf7d-openshift-service-ca\") pod \"4562dca2-c775-43a3-b449-3344e2acaf7d\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " Sep 10 22:38:22.431382 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.431298 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4562dca2-c775-43a3-b449-3344e2acaf7d-maas-api-tls\") pod \"4562dca2-c775-43a3-b449-3344e2acaf7d\" (UID: \"4562dca2-c775-43a3-b449-3344e2acaf7d\") " Sep 10 22:38:22.431654 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.431627 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4562dca2-c775-43a3-b449-3344e2acaf7d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4562dca2-c775-43a3-b449-3344e2acaf7d" (UID: "4562dca2-c775-43a3-b449-3344e2acaf7d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:38:22.433407 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.433380 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4562dca2-c775-43a3-b449-3344e2acaf7d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4562dca2-c775-43a3-b449-3344e2acaf7d" (UID: "4562dca2-c775-43a3-b449-3344e2acaf7d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:38:22.433859 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.433830 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4562dca2-c775-43a3-b449-3344e2acaf7d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4562dca2-c775-43a3-b449-3344e2acaf7d" (UID: "4562dca2-c775-43a3-b449-3344e2acaf7d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:38:22.433959 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.433884 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4562dca2-c775-43a3-b449-3344e2acaf7d-kube-api-access-ljvqc" (OuterVolumeSpecName: "kube-api-access-ljvqc") pod "4562dca2-c775-43a3-b449-3344e2acaf7d" (UID: "4562dca2-c775-43a3-b449-3344e2acaf7d"). InnerVolumeSpecName "kube-api-access-ljvqc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:38:22.532994 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.532882 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4562dca2-c775-43a3-b449-3344e2acaf7d-maas-api-tls\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:38:22.532994 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.532917 2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4562dca2-c775-43a3-b449-3344e2acaf7d-metrics-certs\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:38:22.532994 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.532933 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljvqc\" (UniqueName: \"kubernetes.io/projected/4562dca2-c775-43a3-b449-3344e2acaf7d-kube-api-access-ljvqc\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:38:22.532994 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.532946 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4562dca2-c775-43a3-b449-3344e2acaf7d-openshift-service-ca\") on node \"ip-10-0-139-247.ec2.internal\" DevicePath \"\"" Sep 10 22:38:22.742087 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.742051 2576 generic.go:356] "Generic (PLEG): container finished" podID="4562dca2-c775-43a3-b449-3344e2acaf7d" containerID="a0bb9bf4a2c64ea5c16b4207f1a82cf77ca617667f22f015ff2a9600ec4f8567" exitCode=0 Sep 10 22:38:22.742305 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.742102 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" Sep 10 22:38:22.742305 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.742106 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" event={"ID":"4562dca2-c775-43a3-b449-3344e2acaf7d","Type":"ContainerDied","Data":"a0bb9bf4a2c64ea5c16b4207f1a82cf77ca617667f22f015ff2a9600ec4f8567"} Sep 10 22:38:22.742305 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.742235 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq" event={"ID":"4562dca2-c775-43a3-b449-3344e2acaf7d","Type":"ContainerDied","Data":"cef6c03a3ba168ace03480983c846525772d19e023ae45e076b25d96d8e37f89"} Sep 10 22:38:22.742305 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.742266 2576 scope.go:117] "RemoveContainer" containerID="a0bb9bf4a2c64ea5c16b4207f1a82cf77ca617667f22f015ff2a9600ec4f8567" Sep 10 22:38:22.751728 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.751708 2576 scope.go:117] "RemoveContainer" containerID="a0bb9bf4a2c64ea5c16b4207f1a82cf77ca617667f22f015ff2a9600ec4f8567" Sep 10 22:38:22.752065 ip-10-0-139-247 kubenswrapper[2576]: E0910 22:38:22.752047 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0bb9bf4a2c64ea5c16b4207f1a82cf77ca617667f22f015ff2a9600ec4f8567\": container with ID starting with a0bb9bf4a2c64ea5c16b4207f1a82cf77ca617667f22f015ff2a9600ec4f8567 not found: ID does not exist" containerID="a0bb9bf4a2c64ea5c16b4207f1a82cf77ca617667f22f015ff2a9600ec4f8567" Sep 10 22:38:22.752144 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.752077 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0bb9bf4a2c64ea5c16b4207f1a82cf77ca617667f22f015ff2a9600ec4f8567"} err="failed to get container status \"a0bb9bf4a2c64ea5c16b4207f1a82cf77ca617667f22f015ff2a9600ec4f8567\": rpc error: code = NotFound desc = could not find container \"a0bb9bf4a2c64ea5c16b4207f1a82cf77ca617667f22f015ff2a9600ec4f8567\": container with ID starting with a0bb9bf4a2c64ea5c16b4207f1a82cf77ca617667f22f015ff2a9600ec4f8567 not found: ID does not exist" Sep 10 22:38:22.785972 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.785884 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq"] Sep 10 22:38:22.790673 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:22.790644 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-88khq"] Sep 10 22:38:24.431692 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:24.431659 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4562dca2-c775-43a3-b449-3344e2acaf7d" path="/var/lib/kubelet/pods/4562dca2-c775-43a3-b449-3344e2acaf7d/volumes" Sep 10 22:38:43.074196 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:43.074103 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-85457bf75f-kk854_e605682f-7800-4c7d-9715-c4f1edeed031/manager/0.log" Sep 10 22:38:43.590222 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:43.590196 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76d44b6d5b-v5mwg_22914f3d-9e83-45d0-a503-f995e39dc50c/copy-manifests/0.log" Sep 10 22:38:43.597899 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:43.597869 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76d44b6d5b-v5mwg_22914f3d-9e83-45d0-a503-f995e39dc50c/manager/0.log" Sep 10 22:38:44.184263 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:44.184234 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-559477f47-xclh8_4ecbe2d4-ea45-4e89-8d3e-27ec4807c37e/maas-api/0.log" Sep 10 22:38:44.404203 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:44.404173 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817990-trvrg_24c48036-3df4-4851-9819-308f0a33abb1/cleanup/0.log" Sep 10 22:38:45.816907 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:45.816873 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-4m5fm_33b10c4d-b3a8-4c04-84d3-b9cf8d640d10/manager/0.log" Sep 10 22:38:46.536317 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:46.536282 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-tw2s5_033c5b0a-8df9-4838-9617-5be1317d86ca/discovery/0.log" Sep 10 22:38:47.039213 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:47.039180 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-dbps6_a8c4d41b-f465-41a7-829b-19e4f958a1df/payload-pre-processing/0.log" Sep 10 22:38:47.851916 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:47.851884 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-ww9sp_8ebcd5fb-6401-4b70-a840-e2fdf73d9c57/main/0.log" Sep 10 22:38:47.967721 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:47.967612 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-4t57l_fbddc39e-31bd-4631-ac51-36cd587232c2/main/0.log" Sep 10 22:38:48.191340 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:48.191309 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-bhjwx_d3a1cfb4-9309-422b-b9be-9ee4b7e0e30a/main/0.log" Sep 10 22:38:54.228128 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:54.228092 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:38:55.384950 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:55.384904 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" limit=1024 Sep 10 22:38:57.340026 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:57.339992 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6dx6d_291b13a2-ed89-49f1-98e8-9c9bd3b7f7a7/global-pull-secret-syncer/0.log" Sep 10 22:38:57.486080 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:57.486050 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2vmgg_2a3ae936-7af2-40dc-ac1b-88ed2341dbfe/konnectivity-agent/0.log" Sep 10 22:38:57.665871 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:38:57.665747 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-247.ec2.internal_6969b8244f652594d09b7861b9d8eb44/haproxy/0.log" Sep 10 22:39:02.936517 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:02.936481 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-4m5fm_33b10c4d-b3a8-4c04-84d3-b9cf8d640d10/manager/0.log" Sep 10 22:39:04.803017 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:04.802984 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ed32fc6-5af6-422b-abcd-0639485ebc5d/alertmanager/0.log" Sep 10 22:39:04.836960 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:04.836936 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ed32fc6-5af6-422b-abcd-0639485ebc5d/config-reloader/0.log" Sep 10 22:39:04.868925 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:04.868895 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ed32fc6-5af6-422b-abcd-0639485ebc5d/kube-rbac-proxy-web/0.log" Sep 10 22:39:04.906491 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:04.906459 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ed32fc6-5af6-422b-abcd-0639485ebc5d/kube-rbac-proxy/0.log" Sep 10 22:39:04.946887 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:04.946862 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ed32fc6-5af6-422b-abcd-0639485ebc5d/kube-rbac-proxy-metric/0.log" Sep 10 22:39:04.978153 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:04.978119 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ed32fc6-5af6-422b-abcd-0639485ebc5d/prom-label-proxy/0.log" Sep 10 22:39:05.015397 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:05.015331 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3ed32fc6-5af6-422b-abcd-0639485ebc5d/init-config-reloader/0.log" Sep 10 22:39:05.079860 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:05.079831 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-qbvrr_efc199a6-0151-4410-a431-d8652e0541f4/cluster-monitoring-operator/0.log" Sep 10 22:39:05.118117 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:05.118092 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-nswfg_b7ba4ac6-2237-4cd3-95d9-c23cdea61151/kube-state-metrics/0.log" Sep 10 22:39:05.144463 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:05.144418 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-nswfg_b7ba4ac6-2237-4cd3-95d9-c23cdea61151/kube-rbac-proxy-main/0.log" Sep 10 22:39:05.186251 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:05.186220 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-nswfg_b7ba4ac6-2237-4cd3-95d9-c23cdea61151/kube-rbac-proxy-self/0.log" Sep 10 22:39:05.386640 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:05.386607 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-wswcj_41258563-033c-42e0-a51e-4e77fb6f11c6/monitoring-plugin/0.log" Sep 10 22:39:05.689324 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:05.689289 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rvclz_d5d5cd61-7ac6-4def-bcd7-883ababaa59a/node-exporter/0.log" Sep 10 22:39:05.715143 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:05.715109 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rvclz_d5d5cd61-7ac6-4def-bcd7-883ababaa59a/kube-rbac-proxy/0.log" Sep 10 22:39:05.738062 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:05.738029 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rvclz_d5d5cd61-7ac6-4def-bcd7-883ababaa59a/init-textfile/0.log" Sep 10 22:39:05.873470 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:05.873439 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2c00e9cc-f49a-45f5-a3b2-01bc2146bafa/prometheus/0.log" Sep 10 22:39:05.898021 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:05.897994 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2c00e9cc-f49a-45f5-a3b2-01bc2146bafa/config-reloader/0.log" Sep 10 22:39:05.922929 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:05.922839 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2c00e9cc-f49a-45f5-a3b2-01bc2146bafa/thanos-sidecar/0.log" Sep 10 22:39:05.947981 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:05.947954 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2c00e9cc-f49a-45f5-a3b2-01bc2146bafa/kube-rbac-proxy-web/0.log" Sep 10 22:39:05.971970 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:05.971944 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2c00e9cc-f49a-45f5-a3b2-01bc2146bafa/kube-rbac-proxy/0.log" Sep 10 22:39:05.996696 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:05.996664 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2c00e9cc-f49a-45f5-a3b2-01bc2146bafa/kube-rbac-proxy-thanos/0.log" Sep 10 22:39:06.023430 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:06.023401 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2c00e9cc-f49a-45f5-a3b2-01bc2146bafa/init-config-reloader/0.log" Sep 10 22:39:06.062991 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:06.062957 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-mqrg4_849fcec6-8056-479e-bf59-55d371abb6a1/prometheus-operator/0.log" Sep 10 22:39:06.086609 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:06.086576 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-mqrg4_849fcec6-8056-479e-bf59-55d371abb6a1/kube-rbac-proxy/0.log" Sep 10 22:39:06.122757 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:06.122720 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-dw5xc_0f1f1d9a-4c30-43a5-8f40-2444b200fe9b/prometheus-operator-admission-webhook/0.log" Sep 10 22:39:06.155746 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:06.155718 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c44fd675d-t98kh_ff35ebdf-3271-4d7b-8490-dd9c25d9e714/telemeter-client/0.log" Sep 10 22:39:06.179121 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:06.179048 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c44fd675d-t98kh_ff35ebdf-3271-4d7b-8490-dd9c25d9e714/reload/0.log" Sep 10 22:39:06.202476 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:06.202449 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c44fd675d-t98kh_ff35ebdf-3271-4d7b-8490-dd9c25d9e714/kube-rbac-proxy/0.log" Sep 10 22:39:06.990668 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:06.990570 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz"] Sep 10 22:39:06.991207 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:06.991182 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4562dca2-c775-43a3-b449-3344e2acaf7d" containerName="maas-api" Sep 10 22:39:06.991277 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:06.991209 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4562dca2-c775-43a3-b449-3344e2acaf7d" containerName="maas-api" Sep 10 22:39:06.991337 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:06.991323 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4562dca2-c775-43a3-b449-3344e2acaf7d" containerName="maas-api" Sep 10 22:39:06.994937 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:06.994913 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:06.997276 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:06.997254 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8dpvh\"/\"openshift-service-ca.crt\"" Sep 10 22:39:06.997414 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:06.997321 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8dpvh\"/\"kube-root-ca.crt\"" Sep 10 22:39:06.997854 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:06.997835 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8dpvh\"/\"default-dockercfg-6cqgb\"" Sep 10 22:39:07.005159 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.005130 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz"] Sep 10 22:39:07.061415 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.061371 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/86df6499-70de-4f8c-8f50-c9748d8fae8b-podres\") pod \"perf-node-gather-daemonset-j8zbz\" (UID: \"86df6499-70de-4f8c-8f50-c9748d8fae8b\") " pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.061584 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.061432 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/86df6499-70de-4f8c-8f50-c9748d8fae8b-lib-modules\") pod \"perf-node-gather-daemonset-j8zbz\" (UID: \"86df6499-70de-4f8c-8f50-c9748d8fae8b\") " pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.061584 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.061456 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2qxr\" (UniqueName: \"kubernetes.io/projected/86df6499-70de-4f8c-8f50-c9748d8fae8b-kube-api-access-j2qxr\") pod \"perf-node-gather-daemonset-j8zbz\" (UID: \"86df6499-70de-4f8c-8f50-c9748d8fae8b\") " pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.061584 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.061501 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86df6499-70de-4f8c-8f50-c9748d8fae8b-sys\") pod \"perf-node-gather-daemonset-j8zbz\" (UID: \"86df6499-70de-4f8c-8f50-c9748d8fae8b\") " pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.061584 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.061546 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/86df6499-70de-4f8c-8f50-c9748d8fae8b-proc\") pod \"perf-node-gather-daemonset-j8zbz\" (UID: \"86df6499-70de-4f8c-8f50-c9748d8fae8b\") " pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.162658 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.162612 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/86df6499-70de-4f8c-8f50-c9748d8fae8b-proc\") pod \"perf-node-gather-daemonset-j8zbz\" (UID: \"86df6499-70de-4f8c-8f50-c9748d8fae8b\") " pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.162888 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.162710 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/86df6499-70de-4f8c-8f50-c9748d8fae8b-podres\") pod \"perf-node-gather-daemonset-j8zbz\" (UID: \"86df6499-70de-4f8c-8f50-c9748d8fae8b\") " pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.162888 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.162736 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/86df6499-70de-4f8c-8f50-c9748d8fae8b-lib-modules\") pod \"perf-node-gather-daemonset-j8zbz\" (UID: \"86df6499-70de-4f8c-8f50-c9748d8fae8b\") " pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.162888 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.162754 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2qxr\" (UniqueName: \"kubernetes.io/projected/86df6499-70de-4f8c-8f50-c9748d8fae8b-kube-api-access-j2qxr\") pod \"perf-node-gather-daemonset-j8zbz\" (UID: \"86df6499-70de-4f8c-8f50-c9748d8fae8b\") " pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.162888 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.162754 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/86df6499-70de-4f8c-8f50-c9748d8fae8b-proc\") pod \"perf-node-gather-daemonset-j8zbz\" (UID: \"86df6499-70de-4f8c-8f50-c9748d8fae8b\") " pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.162888 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.162844 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86df6499-70de-4f8c-8f50-c9748d8fae8b-sys\") pod \"perf-node-gather-daemonset-j8zbz\" (UID: \"86df6499-70de-4f8c-8f50-c9748d8fae8b\") " pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.163100 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.162915 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86df6499-70de-4f8c-8f50-c9748d8fae8b-sys\") pod \"perf-node-gather-daemonset-j8zbz\" (UID: \"86df6499-70de-4f8c-8f50-c9748d8fae8b\") " pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.163100 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.162928 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/86df6499-70de-4f8c-8f50-c9748d8fae8b-lib-modules\") pod \"perf-node-gather-daemonset-j8zbz\" (UID: \"86df6499-70de-4f8c-8f50-c9748d8fae8b\") " pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.163100 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.162931 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/86df6499-70de-4f8c-8f50-c9748d8fae8b-podres\") pod \"perf-node-gather-daemonset-j8zbz\" (UID: \"86df6499-70de-4f8c-8f50-c9748d8fae8b\") " pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.171723 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.171685 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2qxr\" (UniqueName: \"kubernetes.io/projected/86df6499-70de-4f8c-8f50-c9748d8fae8b-kube-api-access-j2qxr\") pod \"perf-node-gather-daemonset-j8zbz\" (UID: \"86df6499-70de-4f8c-8f50-c9748d8fae8b\") " pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.306196 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.306096 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.579569 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.579545 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz"] Sep 10 22:39:07.582566 ip-10-0-139-247 kubenswrapper[2576]: W0910 22:39:07.582536 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod86df6499_70de_4f8c_8f50_c9748d8fae8b.slice/crio-9ef22495fc1584ed25850ad2d0aef94a341cc0e5578428dc127bc3ad141101d8 WatchSource:0}: Error finding container 9ef22495fc1584ed25850ad2d0aef94a341cc0e5578428dc127bc3ad141101d8: Status 404 returned error can't find the container with id 9ef22495fc1584ed25850ad2d0aef94a341cc0e5578428dc127bc3ad141101d8 Sep 10 22:39:07.974679 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.974591 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" event={"ID":"86df6499-70de-4f8c-8f50-c9748d8fae8b","Type":"ContainerStarted","Data":"44ca7fa8ed4f57443d2106b1ce171a81dd8f281b8ac0de58c5115472a0974ecf"} Sep 10 22:39:07.974679 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.974645 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" event={"ID":"86df6499-70de-4f8c-8f50-c9748d8fae8b","Type":"ContainerStarted","Data":"9ef22495fc1584ed25850ad2d0aef94a341cc0e5578428dc127bc3ad141101d8"} Sep 10 22:39:07.974899 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.974706 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:07.991639 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:07.991575 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" podStartSLOduration=1.991556105 podStartE2EDuration="1.991556105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:39:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:39:07.989724891 +0000 UTC m=+2278.111423666" watchObservedRunningTime="2026-09-10 22:39:07.991556105 +0000 UTC m=+2278.113254886" Sep 10 22:39:09.408343 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:09.408314 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-ktm6m_39a311e1-4d1f-4d6e-b3de-752d6e620252/volume-data-source-validator/0.log" Sep 10 22:39:10.416884 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:10.416855 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hmlp2_d7e6f82c-4cc2-4610-892d-9e2581c1264a/dns/0.log" Sep 10 22:39:10.439158 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:10.439133 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hmlp2_d7e6f82c-4cc2-4610-892d-9e2581c1264a/kube-rbac-proxy/0.log" Sep 10 22:39:10.564555 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:10.564525 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8z8zh_bd1bec7b-f106-4ca2-b72d-71a7fe7a1929/dns-node-resolver/0.log" Sep 10 22:39:11.369329 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:11.369297 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zfmq5_a95cad0a-1c31-4ecf-915e-05b752bdaeee/node-ca/0.log" Sep 10 22:39:12.712024 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:12.711989 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-tw2s5_033c5b0a-8df9-4838-9617-5be1317d86ca/discovery/0.log" Sep 10 22:39:12.848022 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:12.847992 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-dbps6_a8c4d41b-f465-41a7-829b-19e4f958a1df/payload-pre-processing/0.log" Sep 10 22:39:13.903985 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:13.903953 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7tzf6_9294b995-aea4-496b-8e2c-9739ceda5668/serve-healthcheck-canary/0.log" Sep 10 22:39:13.989346 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:13.989311 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8dpvh/perf-node-gather-daemonset-j8zbz" Sep 10 22:39:14.419751 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:14.419724 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gpnrt_5dd313b9-389a-4562-a835-824681d8e89e/kube-rbac-proxy/0.log" Sep 10 22:39:14.443591 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:14.443557 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gpnrt_5dd313b9-389a-4562-a835-824681d8e89e/exporter/0.log" Sep 10 22:39:14.467388 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:14.467357 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gpnrt_5dd313b9-389a-4562-a835-824681d8e89e/extractor/0.log" Sep 10 22:39:17.121983 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:17.121949 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-85457bf75f-kk854_e605682f-7800-4c7d-9715-c4f1edeed031/manager/0.log" Sep 10 22:39:17.444269 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:17.444193 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76d44b6d5b-v5mwg_22914f3d-9e83-45d0-a503-f995e39dc50c/manager/0.log" Sep 10 22:39:17.467666 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:17.467638 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76d44b6d5b-v5mwg_22914f3d-9e83-45d0-a503-f995e39dc50c/copy-manifests/0.log" Sep 10 22:39:18.912368 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:18.912335 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-tw2s5_033c5b0a-8df9-4838-9617-5be1317d86ca/discovery/0.log" Sep 10 22:39:18.915552 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:18.915528 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-56f58b8775-f8cn7_5687ceb7-b470-42d1-85ac-1b5993671b35/manager/0.log" Sep 10 22:39:19.006850 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:19.006820 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-dbps6_a8c4d41b-f465-41a7-829b-19e4f958a1df/payload-pre-processing/0.log" Sep 10 22:39:20.796045 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:20.796009 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-tw2s5_033c5b0a-8df9-4838-9617-5be1317d86ca/discovery/0.log" Sep 10 22:39:20.888041 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:20.888006 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-dbps6_a8c4d41b-f465-41a7-829b-19e4f958a1df/payload-pre-processing/0.log" Sep 10 22:39:24.620929 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:24.620899 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-2t7c5_1566f5ee-e336-455a-b28c-a77e91708be2/kube-storage-version-migrator-operator/1.log" Sep 10 22:39:24.622110 ip-10-0-139-247 kubenswrapper[2576]: I0910 22:39:24.622085 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-2t7c5_1566f5ee-e336-455a-b28c-a77e91708be2/kube-storage-version-migrator-operator/0.log"