Jul 30 00:43:17.097212 ip-10-0-136-247 systemd[1]: Starting Kubernetes Kubelet... Jul 30 00:43:17.442151 ip-10-0-136-247 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 00:43:17.442151 ip-10-0-136-247 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 30 00:43:17.442151 ip-10-0-136-247 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 00:43:17.442151 ip-10-0-136-247 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 30 00:43:17.442151 ip-10-0-136-247 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 00:43:17.442801 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.442140 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 30 00:43:17.446817 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446800 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 00:43:17.446817 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446816 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446820 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446823 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446826 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446829 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446833 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446836 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446839 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446842 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446844 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446847 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446850 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446853 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446856 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446860 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446864 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446867 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446871 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446874 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 00:43:17.446884 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446877 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446885 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446888 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446891 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446904 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446907 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446910 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446913 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446916 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446918 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446920 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446923 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446925 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446928 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446930 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446933 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446936 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446939 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446941 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446944 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 00:43:17.447362 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446947 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446949 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446953 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446956 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446960 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446963 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446965 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446967 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446970 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446973 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446975 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446978 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446980 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446983 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446987 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446990 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446992 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446995 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.446998 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 00:43:17.447834 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447000 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447003 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447005 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447008 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447010 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447013 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447016 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447018 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447021 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447023 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447027 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447031 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447033 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447036 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447039 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447041 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447044 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447046 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447050 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447052 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 00:43:17.448343 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447056 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447059 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447061 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447064 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447066 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447069 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447071 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447075 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447077 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447080 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447083 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447085 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447088 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447090 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447093 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447095 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447098 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447100 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447103 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447106 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 00:43:17.448819 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447823 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447829 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447833 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447836 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447839 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447842 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447844 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447847 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447850 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447853 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447857 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447861 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447865 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447868 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447871 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447873 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447876 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447878 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447881 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 00:43:17.449311 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447885 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447889 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447907 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447910 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447913 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447916 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447919 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447922 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447924 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447927 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447929 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447933 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447935 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447938 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447940 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447943 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447945 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447948 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447950 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 00:43:17.449776 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447953 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447955 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447958 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447960 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447963 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447966 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447969 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447971 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447974 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447976 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447979 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447981 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447984 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447987 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447990 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447992 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447995 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.447997 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448000 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448002 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 00:43:17.450262 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448005 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448009 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448011 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448014 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448016 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448019 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448021 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448023 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448026 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448029 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448031 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448033 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448036 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448039 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448041 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448043 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448046 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448048 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448051 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448055 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 00:43:17.450770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448057 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448060 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448062 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448065 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448067 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448070 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448073 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448075 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448078 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448080 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448083 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448085 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448088 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448090 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448093 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448096 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448098 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448101 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448103 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448106 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 00:43:17.451275 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448109 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448508 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448513 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448516 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448518 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448521 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448523 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448526 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448529 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448531 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448533 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448536 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448539 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448542 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448545 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448548 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448550 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448553 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448556 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448559 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 00:43:17.451770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448561 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448564 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448567 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448569 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448572 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448574 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448577 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448580 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448582 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448585 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448588 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448590 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448593 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448595 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448598 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448602 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448606 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448608 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448611 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 00:43:17.452273 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448614 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448616 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448620 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448622 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448625 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448628 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448630 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448633 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448635 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448637 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448646 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448648 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448651 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448654 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448657 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448659 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448662 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448664 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448667 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 00:43:17.452763 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448669 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448672 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448674 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448677 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448679 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448682 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448685 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448687 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448690 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448692 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448695 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448697 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448700 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448702 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448705 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448707 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448710 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448712 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448715 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448718 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 00:43:17.453236 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448720 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448723 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448725 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448731 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448733 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448736 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448738 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448741 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448744 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448746 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448749 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448753 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448757 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448760 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448763 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448765 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448768 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448770 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448773 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448775 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 00:43:17.453705 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448778 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.448780 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448862 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448873 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448891 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448916 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448922 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448926 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448933 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448939 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448943 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448946 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448950 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448954 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448957 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448962 2564 flags.go:64] FLAG: --cgroup-root="" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448965 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448968 2564 flags.go:64] FLAG: --client-ca-file="" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448971 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448974 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448980 2564 flags.go:64] FLAG: --cluster-domain="" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448982 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448985 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448988 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 30 00:43:17.454211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448992 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448996 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.448999 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449002 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449005 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449008 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449011 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449014 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449017 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449020 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449024 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449027 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449029 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449032 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449035 2564 flags.go:64] FLAG: --enable-server="true" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449038 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449042 2564 flags.go:64] FLAG: --event-burst="100" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449045 2564 flags.go:64] FLAG: --event-qps="50" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449048 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449051 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449054 2564 flags.go:64] FLAG: --eviction-hard="" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449058 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449061 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449066 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449069 2564 flags.go:64] FLAG: --eviction-soft="" Jul 30 00:43:17.454911 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449072 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449074 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449077 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449080 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449083 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449087 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449090 2564 flags.go:64] FLAG: --feature-gates="" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449094 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449097 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449100 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449103 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449106 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449109 2564 flags.go:64] FLAG: --help="false" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449112 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-136-247.ec2.internal" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449115 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449118 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449121 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449124 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449127 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449130 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449133 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449136 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449138 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 30 00:43:17.455530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449141 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449144 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449147 2564 flags.go:64] FLAG: --kube-reserved="" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449150 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449153 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449155 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449159 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449163 2564 flags.go:64] FLAG: --lock-file="" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449166 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449169 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449172 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449178 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449181 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449184 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449187 2564 flags.go:64] FLAG: --logging-format="text" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449190 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449193 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449196 2564 flags.go:64] FLAG: --manifest-url="" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449199 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449203 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449206 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449210 2564 flags.go:64] FLAG: --max-pods="110" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449213 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449216 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449219 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 30 00:43:17.456142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449222 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449225 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449228 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449231 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449238 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449241 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449244 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449247 2564 flags.go:64] FLAG: --pod-cidr="" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449250 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449255 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449258 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449261 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449264 2564 flags.go:64] FLAG: --port="10250" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449267 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449272 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f4c3a1265b68463b" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449275 2564 flags.go:64] FLAG: --qos-reserved="" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449278 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449281 2564 flags.go:64] FLAG: --register-node="true" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449284 2564 flags.go:64] FLAG: --register-with-taints="" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449287 2564 flags.go:64] FLAG: --registry-burst="10" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449290 2564 flags.go:64] FLAG: --registry-qps="5" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449293 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449295 2564 flags.go:64] FLAG: --reserved-memory="" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449299 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449302 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 30 00:43:17.456780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449305 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449308 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449311 2564 flags.go:64] FLAG: --runonce="false" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449314 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449317 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449320 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449323 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449326 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449329 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449332 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449335 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449337 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449340 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449343 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449346 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449349 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449351 2564 flags.go:64] FLAG: --system-cgroups="" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449354 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449360 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449362 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449365 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449373 2564 flags.go:64] FLAG: --tls-min-version="" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449376 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449379 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449382 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 30 00:43:17.457414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449385 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449388 2564 flags.go:64] FLAG: --v="2" Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449392 2564 flags.go:64] FLAG: --version="false" Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449396 2564 flags.go:64] FLAG: --vmodule="" Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449400 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.449403 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449539 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449542 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449546 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449548 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449551 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449554 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449556 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449559 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449562 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449565 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449567 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449570 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449573 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449575 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449578 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 00:43:17.458041 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449582 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449585 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449587 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449590 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449592 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449595 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449597 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449603 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449605 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449609 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449611 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449614 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449617 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449619 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449622 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449624 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449627 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449629 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449632 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449635 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 00:43:17.458559 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449637 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449640 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449643 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449645 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449648 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449650 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449653 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449656 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449658 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449660 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449665 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449668 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449671 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449674 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449677 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449680 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449682 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449685 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449688 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449694 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 00:43:17.459082 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449696 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449699 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449702 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449704 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449707 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449709 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449712 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449714 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449717 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449721 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449724 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449727 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449730 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449733 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449736 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449738 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449741 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449744 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449746 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 00:43:17.459565 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449749 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449751 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449754 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449757 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449759 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449762 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449764 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449766 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449769 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449771 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449774 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449777 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449781 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449784 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449788 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449792 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449796 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449799 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449801 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 00:43:17.460042 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449804 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 00:43:17.460498 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449806 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 00:43:17.460498 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449809 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 00:43:17.460498 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449811 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 00:43:17.460498 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449814 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 00:43:17.460498 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.449817 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 00:43:17.460498 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.450469 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 00:43:17.460498 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.457847 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 30 00:43:17.460498 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.457865 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 30 00:43:17.460498 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.457879 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 30 00:43:17.460498 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.457888 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 30 00:43:17.460498 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.457960 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 00:43:17.460498 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.457967 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 00:43:17.460498 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.457970 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 00:43:17.460498 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.457973 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 00:43:17.460498 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.457977 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.457980 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.457982 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.457985 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.457988 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.457990 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.457993 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.457997 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458002 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458006 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458010 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458012 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458015 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458018 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458021 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458024 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458027 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458030 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458033 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458036 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 00:43:17.460869 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458039 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458041 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458044 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458047 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458050 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458052 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458055 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458058 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458061 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458063 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458066 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458068 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458071 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458074 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458081 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458087 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458090 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458093 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458096 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458098 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 00:43:17.461429 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458101 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458104 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458106 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458109 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458112 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458114 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458117 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458127 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458131 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458133 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458137 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458139 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458142 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458145 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458147 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458151 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458156 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458160 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458164 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458168 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 00:43:17.461982 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458171 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458173 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458178 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458181 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458184 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458187 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458191 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458194 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458197 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458200 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458202 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458205 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458208 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458211 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458214 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458216 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458219 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458222 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458224 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 00:43:17.462495 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458228 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 00:43:17.462985 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458233 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 00:43:17.462985 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458237 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 00:43:17.462985 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458242 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 00:43:17.462985 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458246 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 00:43:17.462985 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458249 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 00:43:17.462985 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458252 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 00:43:17.462985 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458254 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 30 00:43:17.462985 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458258 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 00:43:17.462985 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458261 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 00:43:17.462985 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458264 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 00:43:17.462985 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458266 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 00:43:17.462985 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458269 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 00:43:17.462985 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458271 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 00:43:17.462985 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458274 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 00:43:17.462985 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458276 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 00:43:17.463357 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.458281 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 00:43:17.463357 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458393 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 00:43:17.463357 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458400 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 00:43:17.463357 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458404 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 00:43:17.463357 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458408 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 00:43:17.463357 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458410 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 00:43:17.463357 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458413 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 00:43:17.463357 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458415 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 00:43:17.463357 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458418 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 00:43:17.463357 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458420 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 00:43:17.463357 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458424 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 00:43:17.463357 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458427 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 00:43:17.463357 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458429 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 00:43:17.463357 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458432 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 00:43:17.463357 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458434 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458436 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458440 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458444 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458448 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458451 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458454 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458456 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458459 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458463 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458467 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458471 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458476 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458480 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458483 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458486 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458488 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458491 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458493 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 00:43:17.463722 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458496 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458499 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458501 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458504 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458506 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458509 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458511 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458513 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458516 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458519 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458521 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458524 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458527 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458529 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458531 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458534 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458537 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458540 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458543 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 00:43:17.464302 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458546 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458551 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458555 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458559 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458562 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458565 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458568 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458570 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458573 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458575 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458578 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458580 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458583 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458585 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458588 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458590 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458593 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458595 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458598 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458600 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 00:43:17.464758 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458603 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458605 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458608 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458611 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458613 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458616 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458618 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458621 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458624 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458629 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458634 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458638 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458642 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458645 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458647 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458650 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458652 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458655 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458657 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 00:43:17.465391 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458662 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 00:43:17.466231 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458665 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 00:43:17.466231 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458668 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 00:43:17.466231 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458670 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 00:43:17.466231 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458673 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 00:43:17.466231 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458675 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 00:43:17.466231 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458678 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 00:43:17.466231 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458680 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 00:43:17.466231 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:17.458683 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 00:43:17.466231 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.458688 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 00:43:17.466231 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.458818 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 30 00:43:17.466231 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.461014 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 30 00:43:17.466231 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.462199 2564 server.go:1019] "Starting client certificate rotation" Jul 30 00:43:17.466231 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.462301 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 00:43:17.466231 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.462335 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 00:43:17.466786 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.466679 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 00:43:17.468968 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.468923 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 00:43:17.476629 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.476407 2564 log.go:25] "Validated CRI v1 runtime API" Jul 30 00:43:17.478601 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.478586 2564 log.go:25] "Validated CRI v1 image API" Jul 30 00:43:17.479485 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.479469 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 30 00:43:17.481774 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.481756 2564 fs.go:135] Filesystem UUIDs: map[5aff05e4-72ef-43c1-82e2-9cc3e128a7e5:/dev/nvme0n1p4 6627e753-3683-4d8c-b4f8-98651679370b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 30 00:43:17.481840 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.481775 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 30 00:43:17.487210 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.487071 2564 manager.go:216] Machine: {Timestamp:2026-07-30 00:43:17.485774066 +0000 UTC m=+0.286731166 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102280 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f78949b5f581a3e3d7f37e969f72e SystemUUID:ec2f7894-9b5f-581a-3e3d-7f37e969f72e BootID:9849791b-640e-4c12-b685-272b96187048 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:82:50:ac:35:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:82:50:ac:35:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:1d:c6:8e:24:fc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 30 00:43:17.487210 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.487206 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 30 00:43:17.487329 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.487299 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 30 00:43:17.490291 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.490264 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 30 00:43:17.490484 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.490292 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-247.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 30 00:43:17.491074 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.491062 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 30 00:43:17.491123 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.491094 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 30 00:43:17.491123 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.491103 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 00:43:17.491730 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.491717 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 00:43:17.492665 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.492655 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 30 00:43:17.493859 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.493841 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 00:43:17.495687 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.495674 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 30 00:43:17.495909 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.495898 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 30 00:43:17.495940 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.495933 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 30 00:43:17.495984 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.495954 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 30 00:43:17.496029 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.495984 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 30 00:43:17.496029 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.495999 2564 kubelet.go:398] "Adding apiserver pod source" Jul 30 00:43:17.496029 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.496011 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 30 00:43:17.496447 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.496434 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 00:43:17.496505 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.496454 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 00:43:17.499155 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.499137 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 30 00:43:17.501743 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.501719 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 30 00:43:17.501829 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.501760 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 30 00:43:17.503565 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.503551 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 30 00:43:17.503640 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.503569 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 30 00:43:17.503640 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.503575 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 30 00:43:17.503640 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.503581 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 30 00:43:17.503640 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.503590 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 30 00:43:17.503640 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.503618 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 30 00:43:17.503640 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.503637 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 30 00:43:17.503640 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.503643 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 30 00:43:17.503853 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.503650 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 30 00:43:17.503853 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.503657 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 30 00:43:17.503853 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.503665 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 30 00:43:17.503853 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.503671 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 30 00:43:17.504366 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.504351 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 30 00:43:17.504410 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.504370 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 30 00:43:17.507874 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.507845 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-247.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 30 00:43:17.508000 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.507963 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 30 00:43:17.508099 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.508081 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-247.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 30 00:43:17.508306 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.508292 2564 server.go:1268] "Started kubelet" Jul 30 00:43:17.508412 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.508377 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 30 00:43:17.508540 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.508481 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 30 00:43:17.508689 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.508659 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 30 00:43:17.509385 ip-10-0-136-247 systemd[1]: Started Kubernetes Kubelet. Jul 30 00:43:17.510633 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.510613 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 30 00:43:17.510785 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.510769 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 30 00:43:17.514687 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.514615 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 30 00:43:17.517748 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.516909 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-247.ec2.internal.18c6ea8de856aa99 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-247.ec2.internal,UID:ip-10-0-136-247.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-247.ec2.internal,},FirstTimestamp:2026-07-30 00:43:17.508262553 +0000 UTC m=+0.309219653,LastTimestamp:2026-07-30 00:43:17.508262553 +0000 UTC m=+0.309219653,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-247.ec2.internal,}" Jul 30 00:43:17.519380 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.519363 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 30 00:43:17.519457 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.519423 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 30 00:43:17.519510 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.519487 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-247.ec2.internal\" not found" Jul 30 00:43:17.520137 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.520107 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 30 00:43:17.520137 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.520109 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 30 00:43:17.520299 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.520147 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 30 00:43:17.520299 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.520237 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 30 00:43:17.520299 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.520245 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 30 00:43:17.520299 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.520247 2564 factory.go:55] Registering systemd factory Jul 30 00:43:17.520299 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.520263 2564 factory.go:223] Registration of the systemd container factory successfully Jul 30 00:43:17.520528 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.520512 2564 factory.go:153] Registering CRI-O factory Jul 30 00:43:17.520605 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.520532 2564 factory.go:223] Registration of the crio container factory successfully Jul 30 00:43:17.520605 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.520587 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 30 00:43:17.520697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.520613 2564 factory.go:103] Registering Raw factory Jul 30 00:43:17.520697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.520629 2564 manager.go:1196] Started watching for new ooms in manager Jul 30 00:43:17.521244 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.521227 2564 manager.go:349] Starting recovery of all containers Jul 30 00:43:17.521860 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.521823 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 30 00:43:17.522196 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.522156 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-247.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 30 00:43:17.534098 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.534075 2564 manager.go:354] Recovery completed Jul 30 00:43:17.535519 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.535354 2564 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 30 00:43:17.538744 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.538730 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 00:43:17.540872 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.540846 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientMemory" Jul 30 00:43:17.540928 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.540887 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 00:43:17.540928 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.540899 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientPID" Jul 30 00:43:17.541415 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.541394 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6zl8p" Jul 30 00:43:17.541486 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.541427 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 30 00:43:17.541486 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.541438 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 30 00:43:17.541486 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.541441 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 00:43:17.541486 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.541457 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 30 00:43:17.542229 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.542215 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientMemory" Jul 30 00:43:17.542294 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.542239 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 00:43:17.542294 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.542250 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientPID" Jul 30 00:43:17.544361 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.544349 2564 policy_none.go:49] "None policy: Start" Jul 30 00:43:17.544407 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.544365 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 30 00:43:17.544407 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.544376 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 30 00:43:17.546535 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.546523 2564 policy_none.go:47] "Start" Jul 30 00:43:17.549450 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.549432 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6zl8p" Jul 30 00:43:17.553715 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.553636 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-247.ec2.internal.18c6ea8dea484089 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-247.ec2.internal,UID:ip-10-0-136-247.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-247.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-247.ec2.internal,},FirstTimestamp:2026-07-30 00:43:17.540872329 +0000 UTC m=+0.341829428,LastTimestamp:2026-07-30 00:43:17.540872329 +0000 UTC m=+0.341829428,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-247.ec2.internal,}" Jul 30 00:43:17.581664 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.581647 2564 manager.go:337] "Starting Device Plugin manager" Jul 30 00:43:17.581773 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.581689 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 30 00:43:17.581773 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.581703 2564 server.go:87] "Starting device plugin registration server" Jul 30 00:43:17.582041 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.582027 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 30 00:43:17.582093 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.582041 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 30 00:43:17.582159 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.582142 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 30 00:43:17.582371 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.582239 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 30 00:43:17.582371 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.582252 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 30 00:43:17.582871 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.582851 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 30 00:43:17.582958 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.582898 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-247.ec2.internal\" not found" Jul 30 00:43:17.660855 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.660802 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 30 00:43:17.663277 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.662075 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 30 00:43:17.663277 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.662122 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 30 00:43:17.663277 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.662148 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 30 00:43:17.663277 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.662184 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 30 00:43:17.665099 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.665080 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 00:43:17.682331 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.682274 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 00:43:17.684259 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.684236 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientMemory" Jul 30 00:43:17.684378 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.684274 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 00:43:17.684378 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.684306 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientPID" Jul 30 00:43:17.684378 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.684343 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-247.ec2.internal" Jul 30 00:43:17.694708 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.694640 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-247.ec2.internal" Jul 30 00:43:17.694708 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.694671 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-247.ec2.internal\": node \"ip-10-0-136-247.ec2.internal\" not found" Jul 30 00:43:17.719056 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.719018 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-247.ec2.internal\" not found" Jul 30 00:43:17.763991 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.763936 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-247.ec2.internal"] Jul 30 00:43:17.764091 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.764042 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 00:43:17.765224 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.765203 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientMemory" Jul 30 00:43:17.765299 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.765240 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 00:43:17.765299 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.765251 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientPID" Jul 30 00:43:17.767845 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.767831 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 00:43:17.768011 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.767995 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal" Jul 30 00:43:17.768053 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.768040 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 00:43:17.768862 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.768843 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientMemory" Jul 30 00:43:17.768944 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.768874 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 00:43:17.768944 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.768898 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientPID" Jul 30 00:43:17.769178 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.769163 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientMemory" Jul 30 00:43:17.769228 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.769194 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 00:43:17.769228 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.769205 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientPID" Jul 30 00:43:17.771615 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.771598 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-247.ec2.internal" Jul 30 00:43:17.771687 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.771675 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 00:43:17.772494 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.772477 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientMemory" Jul 30 00:43:17.772593 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.772513 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 00:43:17.772593 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.772540 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientPID" Jul 30 00:43:17.798052 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.798023 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-247.ec2.internal\" not found" node="ip-10-0-136-247.ec2.internal" Jul 30 00:43:17.803133 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.803110 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-247.ec2.internal\" not found" node="ip-10-0-136-247.ec2.internal" Jul 30 00:43:17.819697 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.819661 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-247.ec2.internal\" not found" Jul 30 00:43:17.822112 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.822088 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3394a8fe621d4462db0fc96b99ef5cf1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal\" (UID: \"3394a8fe621d4462db0fc96b99ef5cf1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal" Jul 30 00:43:17.822222 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.822127 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3394a8fe621d4462db0fc96b99ef5cf1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal\" (UID: \"3394a8fe621d4462db0fc96b99ef5cf1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal" Jul 30 00:43:17.822222 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.822189 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f42d3ae68cadbebcf1e1b0f116456906-config\") pod \"kube-apiserver-proxy-ip-10-0-136-247.ec2.internal\" (UID: \"f42d3ae68cadbebcf1e1b0f116456906\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-247.ec2.internal" Jul 30 00:43:17.924391 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:17.924359 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-247.ec2.internal\" not found" Jul 30 00:43:17.924391 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.924368 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3394a8fe621d4462db0fc96b99ef5cf1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal\" (UID: \"3394a8fe621d4462db0fc96b99ef5cf1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal" Jul 30 00:43:17.924571 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.924398 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3394a8fe621d4462db0fc96b99ef5cf1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal\" (UID: \"3394a8fe621d4462db0fc96b99ef5cf1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal" Jul 30 00:43:17.924571 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.924444 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3394a8fe621d4462db0fc96b99ef5cf1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal\" (UID: \"3394a8fe621d4462db0fc96b99ef5cf1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal" Jul 30 00:43:17.924571 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.924466 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3394a8fe621d4462db0fc96b99ef5cf1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal\" (UID: \"3394a8fe621d4462db0fc96b99ef5cf1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal" Jul 30 00:43:17.924571 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.924476 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f42d3ae68cadbebcf1e1b0f116456906-config\") pod \"kube-apiserver-proxy-ip-10-0-136-247.ec2.internal\" (UID: \"f42d3ae68cadbebcf1e1b0f116456906\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-247.ec2.internal" Jul 30 00:43:17.924571 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:17.924513 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f42d3ae68cadbebcf1e1b0f116456906-config\") pod \"kube-apiserver-proxy-ip-10-0-136-247.ec2.internal\" (UID: \"f42d3ae68cadbebcf1e1b0f116456906\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-247.ec2.internal" Jul 30 00:43:18.025656 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:18.025580 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-247.ec2.internal\" not found" Jul 30 00:43:18.107015 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.106969 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal" Jul 30 00:43:18.107210 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.106978 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-247.ec2.internal" Jul 30 00:43:18.126106 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:18.126079 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-247.ec2.internal\" not found" Jul 30 00:43:18.227250 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:18.227204 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-247.ec2.internal\" not found" Jul 30 00:43:18.327377 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:18.327305 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-247.ec2.internal\" not found" Jul 30 00:43:18.335987 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.335943 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 00:43:18.428182 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:18.428145 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-247.ec2.internal\" not found" Jul 30 00:43:18.464053 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.464020 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 30 00:43:18.464476 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.464222 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 30 00:43:18.464476 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.464243 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 30 00:43:18.528294 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:18.528253 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-247.ec2.internal\" not found" Jul 30 00:43:18.554959 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.554894 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 00:38:18 +0000 UTC" deadline="2028-04-09 20:15:50.338892218 +0000 UTC" Jul 30 00:43:18.554959 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.554950 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14875h32m31.783947385s" Jul 30 00:43:18.628745 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:18.628664 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-247.ec2.internal\" not found" Jul 30 00:43:18.714916 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.714876 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 00:43:18.716651 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.716624 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientMemory" Jul 30 00:43:18.716773 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.716663 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 00:43:18.716773 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.716675 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeHasSufficientPID" Jul 30 00:43:18.729231 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:18.729195 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-247.ec2.internal\" not found" Jul 30 00:43:18.745217 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.745188 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 00:43:18.818482 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:18.818431 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf42d3ae68cadbebcf1e1b0f116456906.slice/crio-8a5c4638f99a1026452d0a57ed86fa4ee38ff27b5b8a87abee8830e071c59d7b WatchSource:0}: Error finding container 8a5c4638f99a1026452d0a57ed86fa4ee38ff27b5b8a87abee8830e071c59d7b: Status 404 returned error can't find the container with id 8a5c4638f99a1026452d0a57ed86fa4ee38ff27b5b8a87abee8830e071c59d7b Jul 30 00:43:18.818746 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:18.818731 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3394a8fe621d4462db0fc96b99ef5cf1.slice/crio-84f8db5a834be91f73ba39acaee929a7ca6a6d4a182a166dd308e9ac2b400b05 WatchSource:0}: Error finding container 84f8db5a834be91f73ba39acaee929a7ca6a6d4a182a166dd308e9ac2b400b05: Status 404 returned error can't find the container with id 84f8db5a834be91f73ba39acaee929a7ca6a6d4a182a166dd308e9ac2b400b05 Jul 30 00:43:18.819926 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.819895 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-247.ec2.internal" Jul 30 00:43:18.824092 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.824044 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 00:43:18.828227 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.828205 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 00:43:18.829181 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.829137 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal" Jul 30 00:43:18.842132 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.842103 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 00:43:18.858966 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:18.858935 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 00:43:19.497532 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.497500 2564 apiserver.go:52] "Watching apiserver" Jul 30 00:43:19.505322 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.505294 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 30 00:43:19.505846 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.505822 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-247.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt","openshift-dns/node-resolver-7xg42","openshift-image-registry/node-ca-xqfqf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal","openshift-multus/multus-additional-cni-plugins-pjkjg","openshift-multus/network-metrics-daemon-4r4tk","kube-system/global-pull-secret-syncer-dj2qr","openshift-cluster-node-tuning-operator/tuned-mshnj","openshift-multus/multus-tvd2g","openshift-network-diagnostics/network-check-target-z66tw","openshift-network-operator/iptables-alerter-jv24z","openshift-ovn-kubernetes/ovnkube-node-kfmfq","kube-system/konnectivity-agent-ds2cm"] Jul 30 00:43:19.508661 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.508633 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:19.508790 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:19.508713 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:19.510956 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.510933 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:19.511079 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:19.511026 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:19.513404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.513379 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7xg42" Jul 30 00:43:19.515968 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.515671 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xqfqf" Jul 30 00:43:19.515968 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.515694 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 30 00:43:19.515968 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.515723 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bq2v9\"" Jul 30 00:43:19.515968 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.515680 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 30 00:43:19.518304 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.517967 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ds2cm" Jul 30 00:43:19.518649 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.518624 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-57cbs\"" Jul 30 00:43:19.518755 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.518631 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 30 00:43:19.518755 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.518723 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 30 00:43:19.518868 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.518774 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 30 00:43:19.519950 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.519921 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 30 00:43:19.520461 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.520441 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mws9x\"" Jul 30 00:43:19.520551 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.520462 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 30 00:43:19.520856 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.520836 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 30 00:43:19.524283 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.521722 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.524283 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.524251 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 30 00:43:19.524624 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.524311 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 30 00:43:19.524624 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.524375 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 30 00:43:19.524624 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.524490 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.525038 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.525016 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 30 00:43:19.525660 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.525637 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zhkb6\"" Jul 30 00:43:19.525959 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.525925 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 30 00:43:19.527021 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.527001 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 30 00:43:19.527246 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.527230 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7hwz4\"" Jul 30 00:43:19.527340 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.527253 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 30 00:43:19.528008 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.527989 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.530569 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.530526 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8m4zf\"" Jul 30 00:43:19.530569 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.530547 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 30 00:43:19.530827 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.530804 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.531400 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.531380 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 30 00:43:19.531499 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.531405 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 30 00:43:19.531499 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.531440 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 00:43:19.531499 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.531460 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 30 00:43:19.531668 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.531589 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 30 00:43:19.531764 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.531747 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 30 00:43:19.533006 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.532824 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 30 00:43:19.533228 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.533206 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 30 00:43:19.533383 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.533206 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lgf47\"" Jul 30 00:43:19.533576 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.533549 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 30 00:43:19.533677 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.533609 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.534040 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534017 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aef50669-7ed3-432c-ad80-62254f6a5ac7-host\") pod \"node-ca-xqfqf\" (UID: \"aef50669-7ed3-432c-ad80-62254f6a5ac7\") " pod="openshift-image-registry/node-ca-xqfqf" Jul 30 00:43:19.534138 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534053 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-run\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.534138 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534078 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61e4710e-5c0b-475d-80d4-db9f1cc78e27-tmp-dir\") pod \"node-resolver-7xg42\" (UID: \"61e4710e-5c0b-475d-80d4-db9f1cc78e27\") " pod="openshift-dns/node-resolver-7xg42" Jul 30 00:43:19.534138 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534106 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0404fd97-1663-44e4-bf04-6a588423f8ed-system-cni-dir\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.534303 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534156 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0404fd97-1663-44e4-bf04-6a588423f8ed-os-release\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.534303 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534215 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shvhr\" (UniqueName: \"kubernetes.io/projected/4d304196-189e-44d8-ab5e-27ddbb8e18c2-kube-api-access-shvhr\") pod \"network-metrics-daemon-4r4tk\" (UID: \"4d304196-189e-44d8-ab5e-27ddbb8e18c2\") " pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:19.534303 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534241 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfnjb\" (UniqueName: \"kubernetes.io/projected/aef50669-7ed3-432c-ad80-62254f6a5ac7-kube-api-access-zfnjb\") pod \"node-ca-xqfqf\" (UID: \"aef50669-7ed3-432c-ad80-62254f6a5ac7\") " pod="openshift-image-registry/node-ca-xqfqf" Jul 30 00:43:19.534303 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534269 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-sysconfig\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.534303 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534294 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjgm2\" (UniqueName: \"kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2\") pod \"network-check-target-z66tw\" (UID: \"0eb4b506-e95c-452e-a2f1-a0b683231704\") " pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:19.534558 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534333 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/61e4710e-5c0b-475d-80d4-db9f1cc78e27-hosts-file\") pod \"node-resolver-7xg42\" (UID: \"61e4710e-5c0b-475d-80d4-db9f1cc78e27\") " pod="openshift-dns/node-resolver-7xg42" Jul 30 00:43:19.534558 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534358 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0404fd97-1663-44e4-bf04-6a588423f8ed-cnibin\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.534558 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534384 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0404fd97-1663-44e4-bf04-6a588423f8ed-cni-binary-copy\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.534558 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534409 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0404fd97-1663-44e4-bf04-6a588423f8ed-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.534558 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534429 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs\") pod \"network-metrics-daemon-4r4tk\" (UID: \"4d304196-189e-44d8-ab5e-27ddbb8e18c2\") " pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:19.534558 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534450 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/aef50669-7ed3-432c-ad80-62254f6a5ac7-serviceca\") pod \"node-ca-xqfqf\" (UID: \"aef50669-7ed3-432c-ad80-62254f6a5ac7\") " pod="openshift-image-registry/node-ca-xqfqf" Jul 30 00:43:19.534558 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534474 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/84f859e4-75b2-4f4d-b6af-292c6b5fe003-konnectivity-ca\") pod \"konnectivity-agent-ds2cm\" (UID: \"84f859e4-75b2-4f4d-b6af-292c6b5fe003\") " pod="kube-system/konnectivity-agent-ds2cm" Jul 30 00:43:19.534558 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534510 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btb7x\" (UniqueName: \"kubernetes.io/projected/0404fd97-1663-44e4-bf04-6a588423f8ed-kube-api-access-btb7x\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.534558 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534547 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-host\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.535105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534581 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-tmp\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.535105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534600 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0404fd97-1663-44e4-bf04-6a588423f8ed-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.535105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534617 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-modprobe-d\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.535105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534636 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-kubernetes\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.535105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534658 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-sys\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.535105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534684 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k25b\" (UniqueName: \"kubernetes.io/projected/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-kube-api-access-8k25b\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.535105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534707 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-sysctl-conf\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.535105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534750 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-var-lib-kubelet\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.535105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534775 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-tuned\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.535105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534809 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0404fd97-1663-44e4-bf04-6a588423f8ed-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.535105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534835 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/84f859e4-75b2-4f4d-b6af-292c6b5fe003-agent-certs\") pod \"konnectivity-agent-ds2cm\" (UID: \"84f859e4-75b2-4f4d-b6af-292c6b5fe003\") " pod="kube-system/konnectivity-agent-ds2cm" Jul 30 00:43:19.535105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534872 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-sysctl-d\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.535105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534922 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-systemd\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.535105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534946 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-lib-modules\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.535105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.534989 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm5cl\" (UniqueName: \"kubernetes.io/projected/61e4710e-5c0b-475d-80d4-db9f1cc78e27-kube-api-access-fm5cl\") pod \"node-resolver-7xg42\" (UID: \"61e4710e-5c0b-475d-80d4-db9f1cc78e27\") " pod="openshift-dns/node-resolver-7xg42" Jul 30 00:43:19.535861 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.535475 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-52xds\"" Jul 30 00:43:19.535861 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.535674 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 30 00:43:19.536674 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.536647 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:19.536784 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:19.536725 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:19.539294 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.539269 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jv24z" Jul 30 00:43:19.541870 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.541848 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 30 00:43:19.541988 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.541857 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6nwn8\"" Jul 30 00:43:19.541988 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.541898 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 30 00:43:19.541988 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.541919 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 30 00:43:19.553207 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.553180 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-knlb2" Jul 30 00:43:19.561925 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.561867 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-knlb2" Jul 30 00:43:19.621067 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.621037 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 30 00:43:19.635848 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.635813 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-multus-socket-dir-parent\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.635848 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.635852 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-cni-netd\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.636183 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.635877 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-etc-selinux\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.636183 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.635926 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-sysctl-conf\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.636183 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.635952 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-os-release\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.636183 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.635976 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-var-lib-cni-bin\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.636183 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636012 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-run-openvswitch\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.636183 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636035 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-run-ovn\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.636183 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636061 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/84f859e4-75b2-4f4d-b6af-292c6b5fe003-agent-certs\") pod \"konnectivity-agent-ds2cm\" (UID: \"84f859e4-75b2-4f4d-b6af-292c6b5fe003\") " pod="kube-system/konnectivity-agent-ds2cm" Jul 30 00:43:19.636183 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636082 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-lib-modules\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.636183 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636103 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:19.636183 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636126 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-var-lib-openvswitch\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.636183 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636150 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-node-log\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.636183 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636175 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-sys-fs\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.636183 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636177 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-sysctl-conf\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636201 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aef50669-7ed3-432c-ad80-62254f6a5ac7-host\") pod \"node-ca-xqfqf\" (UID: \"aef50669-7ed3-432c-ad80-62254f6a5ac7\") " pod="openshift-image-registry/node-ca-xqfqf" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636224 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-run\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636249 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0404fd97-1663-44e4-bf04-6a588423f8ed-system-cni-dir\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636270 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0404fd97-1663-44e4-bf04-6a588423f8ed-os-release\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636303 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0404fd97-1663-44e4-bf04-6a588423f8ed-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636335 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-lib-modules\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636338 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-systemd-units\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636379 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-host\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636406 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-tmp\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636425 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aef50669-7ed3-432c-ad80-62254f6a5ac7-host\") pod \"node-ca-xqfqf\" (UID: \"aef50669-7ed3-432c-ad80-62254f6a5ac7\") " pod="openshift-image-registry/node-ca-xqfqf" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636435 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prmjq\" (UniqueName: \"kubernetes.io/projected/06214db9-74fe-4a20-986e-321dab2426c7-kube-api-access-prmjq\") pod \"iptables-alerter-jv24z\" (UID: \"06214db9-74fe-4a20-986e-321dab2426c7\") " pod="openshift-network-operator/iptables-alerter-jv24z" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636463 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-multus-conf-dir\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636491 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-etc-kubernetes\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636479 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-run\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636547 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0404fd97-1663-44e4-bf04-6a588423f8ed-system-cni-dir\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636589 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-host\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.636810 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636666 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0404fd97-1663-44e4-bf04-6a588423f8ed-os-release\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636806 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0404fd97-1663-44e4-bf04-6a588423f8ed-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636844 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-modprobe-d\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636872 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-sys\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636887 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636899 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/06214db9-74fe-4a20-986e-321dab2426c7-iptables-alerter-script\") pod \"iptables-alerter-jv24z\" (UID: \"06214db9-74fe-4a20-986e-321dab2426c7\") " pod="openshift-network-operator/iptables-alerter-jv24z" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636943 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-cnibin\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636965 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-run-netns\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.636987 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/34b16659-933b-4d38-86d9-f52957eb9128-env-overrides\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637010 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-registration-dir\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637031 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-run-k8s-cni-cncf-io\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637052 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-hostroot\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637075 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-kubelet\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637103 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-socket-dir\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637125 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-device-dir\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637147 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77pkk\" (UniqueName: \"kubernetes.io/projected/de873ad8-b46e-4dc3-b179-2368592e10e6-kube-api-access-77pkk\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637176 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-var-lib-kubelet\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.637697 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637202 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-tuned\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637228 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0404fd97-1663-44e4-bf04-6a588423f8ed-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637254 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-var-lib-kubelet\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637282 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637309 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-sysctl-d\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637331 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-systemd\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637355 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm5cl\" (UniqueName: \"kubernetes.io/projected/61e4710e-5c0b-475d-80d4-db9f1cc78e27-kube-api-access-fm5cl\") pod \"node-resolver-7xg42\" (UID: \"61e4710e-5c0b-475d-80d4-db9f1cc78e27\") " pod="openshift-dns/node-resolver-7xg42" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637380 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-run-netns\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637406 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-run-ovn-kubernetes\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637431 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/34b16659-933b-4d38-86d9-f52957eb9128-ovnkube-config\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637456 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61e4710e-5c0b-475d-80d4-db9f1cc78e27-tmp-dir\") pod \"node-resolver-7xg42\" (UID: \"61e4710e-5c0b-475d-80d4-db9f1cc78e27\") " pod="openshift-dns/node-resolver-7xg42" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637480 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1fca91ee-9cbd-4da1-b318-96ab379123e0-multus-daemon-config\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637511 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cjbv\" (UniqueName: \"kubernetes.io/projected/1fca91ee-9cbd-4da1-b318-96ab379123e0-kube-api-access-6cjbv\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637539 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/34b16659-933b-4d38-86d9-f52957eb9128-ovn-node-metrics-cert\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637569 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shvhr\" (UniqueName: \"kubernetes.io/projected/4d304196-189e-44d8-ab5e-27ddbb8e18c2-kube-api-access-shvhr\") pod \"network-metrics-daemon-4r4tk\" (UID: \"4d304196-189e-44d8-ab5e-27ddbb8e18c2\") " pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637606 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfnjb\" (UniqueName: \"kubernetes.io/projected/aef50669-7ed3-432c-ad80-62254f6a5ac7-kube-api-access-zfnjb\") pod \"node-ca-xqfqf\" (UID: \"aef50669-7ed3-432c-ad80-62254f6a5ac7\") " pod="openshift-image-registry/node-ca-xqfqf" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637631 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-sysconfig\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.638728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637661 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjgm2\" (UniqueName: \"kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2\") pod \"network-check-target-z66tw\" (UID: \"0eb4b506-e95c-452e-a2f1-a0b683231704\") " pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637683 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/61e4710e-5c0b-475d-80d4-db9f1cc78e27-hosts-file\") pod \"node-resolver-7xg42\" (UID: \"61e4710e-5c0b-475d-80d4-db9f1cc78e27\") " pod="openshift-dns/node-resolver-7xg42" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637705 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0404fd97-1663-44e4-bf04-6a588423f8ed-cnibin\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637727 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0404fd97-1663-44e4-bf04-6a588423f8ed-cni-binary-copy\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637753 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-system-cni-dir\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637779 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs\") pod \"network-metrics-daemon-4r4tk\" (UID: \"4d304196-189e-44d8-ab5e-27ddbb8e18c2\") " pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637781 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-sysctl-d\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637804 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/aef50669-7ed3-432c-ad80-62254f6a5ac7-serviceca\") pod \"node-ca-xqfqf\" (UID: \"aef50669-7ed3-432c-ad80-62254f6a5ac7\") " pod="openshift-image-registry/node-ca-xqfqf" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637840 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/84f859e4-75b2-4f4d-b6af-292c6b5fe003-konnectivity-ca\") pod \"konnectivity-agent-ds2cm\" (UID: \"84f859e4-75b2-4f4d-b6af-292c6b5fe003\") " pod="kube-system/konnectivity-agent-ds2cm" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.637949 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0404fd97-1663-44e4-bf04-6a588423f8ed-cnibin\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638049 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-systemd\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638347 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btb7x\" (UniqueName: \"kubernetes.io/projected/0404fd97-1663-44e4-bf04-6a588423f8ed-kube-api-access-btb7x\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638385 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-kubelet-config\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638396 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/aef50669-7ed3-432c-ad80-62254f6a5ac7-serviceca\") pod \"node-ca-xqfqf\" (UID: \"aef50669-7ed3-432c-ad80-62254f6a5ac7\") " pod="openshift-image-registry/node-ca-xqfqf" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638413 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-multus-cni-dir\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638443 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-var-lib-cni-multus\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638473 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-log-socket\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.639878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638481 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-var-lib-kubelet\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0404fd97-1663-44e4-bf04-6a588423f8ed-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638532 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-slash\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638555 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/34b16659-933b-4d38-86d9-f52957eb9128-ovnkube-script-lib\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638581 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n7kx\" (UniqueName: \"kubernetes.io/projected/34b16659-933b-4d38-86d9-f52957eb9128-kube-api-access-5n7kx\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638606 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638632 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-kubernetes\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638659 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k25b\" (UniqueName: \"kubernetes.io/projected/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-kube-api-access-8k25b\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638685 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-dbus\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638798 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/61e4710e-5c0b-475d-80d4-db9f1cc78e27-hosts-file\") pod \"node-resolver-7xg42\" (UID: \"61e4710e-5c0b-475d-80d4-db9f1cc78e27\") " pod="openshift-dns/node-resolver-7xg42" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638834 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/06214db9-74fe-4a20-986e-321dab2426c7-host-slash\") pod \"iptables-alerter-jv24z\" (UID: \"06214db9-74fe-4a20-986e-321dab2426c7\") " pod="openshift-network-operator/iptables-alerter-jv24z" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638863 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-run-multus-certs\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638893 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-run-systemd\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638919 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-etc-openvswitch\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638945 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-cni-bin\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.638996 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1fca91ee-9cbd-4da1-b318-96ab379123e0-cni-binary-copy\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.639141 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-modprobe-d\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.640894 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.639281 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0404fd97-1663-44e4-bf04-6a588423f8ed-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.641887 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.639641 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/84f859e4-75b2-4f4d-b6af-292c6b5fe003-konnectivity-ca\") pod \"konnectivity-agent-ds2cm\" (UID: \"84f859e4-75b2-4f4d-b6af-292c6b5fe003\") " pod="kube-system/konnectivity-agent-ds2cm" Jul 30 00:43:19.641887 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.639707 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-sys\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.641887 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.639989 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61e4710e-5c0b-475d-80d4-db9f1cc78e27-tmp-dir\") pod \"node-resolver-7xg42\" (UID: \"61e4710e-5c0b-475d-80d4-db9f1cc78e27\") " pod="openshift-dns/node-resolver-7xg42" Jul 30 00:43:19.641887 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.640170 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0404fd97-1663-44e4-bf04-6a588423f8ed-cni-binary-copy\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.641887 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.640233 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-sysconfig\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.641887 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:19.640303 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:43:19.641887 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.640393 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-kubernetes\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.641887 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:19.640760 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs podName:4d304196-189e-44d8-ab5e-27ddbb8e18c2 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:20.140733629 +0000 UTC m=+2.941690743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs") pod "network-metrics-daemon-4r4tk" (UID: "4d304196-189e-44d8-ab5e-27ddbb8e18c2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:43:19.641887 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.640883 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0404fd97-1663-44e4-bf04-6a588423f8ed-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.646728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.646689 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/84f859e4-75b2-4f4d-b6af-292c6b5fe003-agent-certs\") pod \"konnectivity-agent-ds2cm\" (UID: \"84f859e4-75b2-4f4d-b6af-292c6b5fe003\") " pod="kube-system/konnectivity-agent-ds2cm" Jul 30 00:43:19.647196 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:19.647172 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 00:43:19.647347 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:19.647202 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 00:43:19.647347 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:19.647218 2564 projected.go:196] Error preparing data for projected volume kube-api-access-vjgm2 for pod openshift-network-diagnostics/network-check-target-z66tw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:43:19.647347 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:19.647291 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2 podName:0eb4b506-e95c-452e-a2f1-a0b683231704 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:20.147270709 +0000 UTC m=+2.948227795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vjgm2" (UniqueName: "kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2") pod "network-check-target-z66tw" (UID: "0eb4b506-e95c-452e-a2f1-a0b683231704") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:43:19.647533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.647453 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-etc-tuned\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.647672 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.647646 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-tmp\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.650880 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.650838 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shvhr\" (UniqueName: \"kubernetes.io/projected/4d304196-189e-44d8-ab5e-27ddbb8e18c2-kube-api-access-shvhr\") pod \"network-metrics-daemon-4r4tk\" (UID: \"4d304196-189e-44d8-ab5e-27ddbb8e18c2\") " pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:19.651353 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.651332 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm5cl\" (UniqueName: \"kubernetes.io/projected/61e4710e-5c0b-475d-80d4-db9f1cc78e27-kube-api-access-fm5cl\") pod \"node-resolver-7xg42\" (UID: \"61e4710e-5c0b-475d-80d4-db9f1cc78e27\") " pod="openshift-dns/node-resolver-7xg42" Jul 30 00:43:19.654045 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.654015 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k25b\" (UniqueName: \"kubernetes.io/projected/db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0-kube-api-access-8k25b\") pod \"tuned-mshnj\" (UID: \"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0\") " pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.654262 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.654202 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btb7x\" (UniqueName: \"kubernetes.io/projected/0404fd97-1663-44e4-bf04-6a588423f8ed-kube-api-access-btb7x\") pod \"multus-additional-cni-plugins-pjkjg\" (UID: \"0404fd97-1663-44e4-bf04-6a588423f8ed\") " pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.655390 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.655344 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfnjb\" (UniqueName: \"kubernetes.io/projected/aef50669-7ed3-432c-ad80-62254f6a5ac7-kube-api-access-zfnjb\") pod \"node-ca-xqfqf\" (UID: \"aef50669-7ed3-432c-ad80-62254f6a5ac7\") " pod="openshift-image-registry/node-ca-xqfqf" Jul 30 00:43:19.669683 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.669618 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal" event={"ID":"3394a8fe621d4462db0fc96b99ef5cf1","Type":"ContainerStarted","Data":"84f8db5a834be91f73ba39acaee929a7ca6a6d4a182a166dd308e9ac2b400b05"} Jul 30 00:43:19.672528 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.672489 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-247.ec2.internal" event={"ID":"f42d3ae68cadbebcf1e1b0f116456906","Type":"ContainerStarted","Data":"8a5c4638f99a1026452d0a57ed86fa4ee38ff27b5b8a87abee8830e071c59d7b"} Jul 30 00:43:19.740024 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.739985 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-socket-dir\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.740232 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740047 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-device-dir\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.740232 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740075 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77pkk\" (UniqueName: \"kubernetes.io/projected/de873ad8-b46e-4dc3-b179-2368592e10e6-kube-api-access-77pkk\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.740232 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740106 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-var-lib-kubelet\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.740232 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740132 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.740232 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740157 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-socket-dir\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.740232 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740162 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-run-netns\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.740232 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740210 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-run-netns\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.740232 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740222 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-run-ovn-kubernetes\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.740671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740263 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-var-lib-kubelet\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.740671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740263 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-run-ovn-kubernetes\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.740671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740323 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.740671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740350 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/34b16659-933b-4d38-86d9-f52957eb9128-ovnkube-config\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.740671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740380 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1fca91ee-9cbd-4da1-b318-96ab379123e0-multus-daemon-config\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.740671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740407 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cjbv\" (UniqueName: \"kubernetes.io/projected/1fca91ee-9cbd-4da1-b318-96ab379123e0-kube-api-access-6cjbv\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.740671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740432 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/34b16659-933b-4d38-86d9-f52957eb9128-ovn-node-metrics-cert\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.740671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740448 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-device-dir\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.740671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740479 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-system-cni-dir\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.740671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740523 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-kubelet-config\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:19.740671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740551 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-multus-cni-dir\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.740671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740576 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-var-lib-cni-multus\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.740671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740600 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-log-socket\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.740671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740626 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-slash\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.740671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740652 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/34b16659-933b-4d38-86d9-f52957eb9128-ovnkube-script-lib\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740681 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n7kx\" (UniqueName: \"kubernetes.io/projected/34b16659-933b-4d38-86d9-f52957eb9128-kube-api-access-5n7kx\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740706 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740732 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-dbus\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740757 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/06214db9-74fe-4a20-986e-321dab2426c7-host-slash\") pod \"iptables-alerter-jv24z\" (UID: \"06214db9-74fe-4a20-986e-321dab2426c7\") " pod="openshift-network-operator/iptables-alerter-jv24z" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740780 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-run-multus-certs\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740793 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-var-lib-cni-multus\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740805 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-run-systemd\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740832 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-etc-openvswitch\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740883 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-etc-openvswitch\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740925 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-log-socket\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.740963 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-slash\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741076 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1fca91ee-9cbd-4da1-b318-96ab379123e0-multus-daemon-config\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741140 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-system-cni-dir\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741194 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-kubelet-config\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741338 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-multus-cni-dir\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741374 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-cni-bin\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741402 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1fca91ee-9cbd-4da1-b318-96ab379123e0-cni-binary-copy\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.741533 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741427 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-multus-socket-dir-parent\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741452 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-cni-netd\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741477 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-etc-selinux\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741479 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-cni-bin\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741505 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-os-release\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741529 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-var-lib-cni-bin\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741554 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-run-openvswitch\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741580 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-run-ovn\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741609 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741628 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-dbus\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741681 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/34b16659-933b-4d38-86d9-f52957eb9128-ovnkube-script-lib\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741694 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-var-lib-openvswitch\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741635 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-var-lib-openvswitch\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741748 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-run-multus-certs\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741754 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-node-log\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741763 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-run-systemd\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741792 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-sys-fs\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741809 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/06214db9-74fe-4a20-986e-321dab2426c7-host-slash\") pod \"iptables-alerter-jv24z\" (UID: \"06214db9-74fe-4a20-986e-321dab2426c7\") " pod="openshift-network-operator/iptables-alerter-jv24z" Jul 30 00:43:19.742443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741793 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-node-log\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741830 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-systemd-units\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741853 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prmjq\" (UniqueName: \"kubernetes.io/projected/06214db9-74fe-4a20-986e-321dab2426c7-kube-api-access-prmjq\") pod \"iptables-alerter-jv24z\" (UID: \"06214db9-74fe-4a20-986e-321dab2426c7\") " pod="openshift-network-operator/iptables-alerter-jv24z" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741857 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741870 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-multus-conf-dir\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741887 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-etc-kubernetes\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741906 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/06214db9-74fe-4a20-986e-321dab2426c7-iptables-alerter-script\") pod \"iptables-alerter-jv24z\" (UID: \"06214db9-74fe-4a20-986e-321dab2426c7\") " pod="openshift-network-operator/iptables-alerter-jv24z" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741912 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-sys-fs\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741923 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-cnibin\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741957 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-cnibin\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.741995 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-multus-conf-dir\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742019 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-etc-kubernetes\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742138 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1fca91ee-9cbd-4da1-b318-96ab379123e0-cni-binary-copy\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742203 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-cni-netd\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742230 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-run-netns\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742242 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-systemd-units\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742289 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-run-netns\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742335 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-run-openvswitch\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.743369 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742344 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-etc-selinux\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742362 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/34b16659-933b-4d38-86d9-f52957eb9128-ovnkube-config\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742379 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/34b16659-933b-4d38-86d9-f52957eb9128-env-overrides\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742392 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-os-release\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742419 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-registration-dir\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742435 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-var-lib-cni-bin\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742450 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-run-k8s-cni-cncf-io\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742439 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-run-ovn\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742477 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-hostroot\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-kubelet\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:19.742547 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742579 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/34b16659-933b-4d38-86d9-f52957eb9128-host-kubelet\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:19.742606 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret podName:962582df-c2b5-4b1f-990a-18b7a1c3ccfd nodeName:}" failed. No retries permitted until 2026-07-30 00:43:20.242584428 +0000 UTC m=+3.043541517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret") pod "global-pull-secret-syncer-dj2qr" (UID: "962582df-c2b5-4b1f-990a-18b7a1c3ccfd") : object "kube-system"/"original-pull-secret" not registered Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742665 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/de873ad8-b46e-4dc3-b179-2368592e10e6-registration-dir\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742709 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-host-run-k8s-cni-cncf-io\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742203 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-multus-socket-dir-parent\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742748 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1fca91ee-9cbd-4da1-b318-96ab379123e0-hostroot\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.744254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.742789 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/34b16659-933b-4d38-86d9-f52957eb9128-env-overrides\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.745107 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.743187 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/06214db9-74fe-4a20-986e-321dab2426c7-iptables-alerter-script\") pod \"iptables-alerter-jv24z\" (UID: \"06214db9-74fe-4a20-986e-321dab2426c7\") " pod="openshift-network-operator/iptables-alerter-jv24z" Jul 30 00:43:19.745107 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.743614 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/34b16659-933b-4d38-86d9-f52957eb9128-ovn-node-metrics-cert\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.753418 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.749819 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77pkk\" (UniqueName: \"kubernetes.io/projected/de873ad8-b46e-4dc3-b179-2368592e10e6-kube-api-access-77pkk\") pod \"aws-ebs-csi-driver-node-z55qt\" (UID: \"de873ad8-b46e-4dc3-b179-2368592e10e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.753905 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.753855 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prmjq\" (UniqueName: \"kubernetes.io/projected/06214db9-74fe-4a20-986e-321dab2426c7-kube-api-access-prmjq\") pod \"iptables-alerter-jv24z\" (UID: \"06214db9-74fe-4a20-986e-321dab2426c7\") " pod="openshift-network-operator/iptables-alerter-jv24z" Jul 30 00:43:19.753905 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.753883 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n7kx\" (UniqueName: \"kubernetes.io/projected/34b16659-933b-4d38-86d9-f52957eb9128-kube-api-access-5n7kx\") pod \"ovnkube-node-kfmfq\" (UID: \"34b16659-933b-4d38-86d9-f52957eb9128\") " pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.753905 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.753895 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cjbv\" (UniqueName: \"kubernetes.io/projected/1fca91ee-9cbd-4da1-b318-96ab379123e0-kube-api-access-6cjbv\") pod \"multus-tvd2g\" (UID: \"1fca91ee-9cbd-4da1-b318-96ab379123e0\") " pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.829081 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.829033 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7xg42" Jul 30 00:43:19.838334 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.838127 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ds2cm" Jul 30 00:43:19.843695 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:19.843661 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61e4710e_5c0b_475d_80d4_db9f1cc78e27.slice/crio-dec2cbe15816559be20d730afecfada15e967e71de24bd0f9396b1671e27ec18 WatchSource:0}: Error finding container dec2cbe15816559be20d730afecfada15e967e71de24bd0f9396b1671e27ec18: Status 404 returned error can't find the container with id dec2cbe15816559be20d730afecfada15e967e71de24bd0f9396b1671e27ec18 Jul 30 00:43:19.846598 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.846572 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xqfqf" Jul 30 00:43:19.853354 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.853275 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pjkjg" Jul 30 00:43:19.856545 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:19.856516 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84f859e4_75b2_4f4d_b6af_292c6b5fe003.slice/crio-44d742fdfbae5f4dc113d3c646767d9bf93f430cb08f6c9a371363360d8953ef WatchSource:0}: Error finding container 44d742fdfbae5f4dc113d3c646767d9bf93f430cb08f6c9a371363360d8953ef: Status 404 returned error can't find the container with id 44d742fdfbae5f4dc113d3c646767d9bf93f430cb08f6c9a371363360d8953ef Jul 30 00:43:19.859988 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:19.859955 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaef50669_7ed3_432c_ad80_62254f6a5ac7.slice/crio-1eb9dd5c2a8c55c261214b37e7161888d53461a36e7a80e97f699861b060c148 WatchSource:0}: Error finding container 1eb9dd5c2a8c55c261214b37e7161888d53461a36e7a80e97f699861b060c148: Status 404 returned error can't find the container with id 1eb9dd5c2a8c55c261214b37e7161888d53461a36e7a80e97f699861b060c148 Jul 30 00:43:19.861171 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.861145 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mshnj" Jul 30 00:43:19.868745 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:19.868716 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0404fd97_1663_44e4_bf04_6a588423f8ed.slice/crio-3503638cf5bd4cb7298ed824a3c034508ee8ebffdfb3d197c1d90f20e112ff31 WatchSource:0}: Error finding container 3503638cf5bd4cb7298ed824a3c034508ee8ebffdfb3d197c1d90f20e112ff31: Status 404 returned error can't find the container with id 3503638cf5bd4cb7298ed824a3c034508ee8ebffdfb3d197c1d90f20e112ff31 Jul 30 00:43:19.870487 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.870464 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:19.876176 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:19.876144 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb531aba_6f1e_4ab2_b7c3_3b4c2cf7b0a0.slice/crio-4fba1f9863a8de43c3b82bd10b8e892a5f17fb75f20aabb2eef3aa5547421d46 WatchSource:0}: Error finding container 4fba1f9863a8de43c3b82bd10b8e892a5f17fb75f20aabb2eef3aa5547421d46: Status 404 returned error can't find the container with id 4fba1f9863a8de43c3b82bd10b8e892a5f17fb75f20aabb2eef3aa5547421d46 Jul 30 00:43:19.879414 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.879391 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" Jul 30 00:43:19.885229 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:19.885193 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b16659_933b_4d38_86d9_f52957eb9128.slice/crio-cb984cf06640d03254f1db8cf4349aefb98af536ff60eb96360501eba822a5dc WatchSource:0}: Error finding container cb984cf06640d03254f1db8cf4349aefb98af536ff60eb96360501eba822a5dc: Status 404 returned error can't find the container with id cb984cf06640d03254f1db8cf4349aefb98af536ff60eb96360501eba822a5dc Jul 30 00:43:19.887394 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.887368 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tvd2g" Jul 30 00:43:19.893308 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.893101 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jv24z" Jul 30 00:43:19.905637 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:19.905596 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fca91ee_9cbd_4da1_b318_96ab379123e0.slice/crio-e3d2cb2a7bdada111b5af72950f681ceee7928c0d7f9cf655a561eeeb3cf0102 WatchSource:0}: Error finding container e3d2cb2a7bdada111b5af72950f681ceee7928c0d7f9cf655a561eeeb3cf0102: Status 404 returned error can't find the container with id e3d2cb2a7bdada111b5af72950f681ceee7928c0d7f9cf655a561eeeb3cf0102 Jul 30 00:43:19.909246 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:19.909216 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde873ad8_b46e_4dc3_b179_2368592e10e6.slice/crio-b3254c27de4f5268b66643782fc9c7272acd0f8660dd9a0eb0bd5af0f54e0ce6 WatchSource:0}: Error finding container b3254c27de4f5268b66643782fc9c7272acd0f8660dd9a0eb0bd5af0f54e0ce6: Status 404 returned error can't find the container with id b3254c27de4f5268b66643782fc9c7272acd0f8660dd9a0eb0bd5af0f54e0ce6 Jul 30 00:43:19.911555 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:19.911526 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06214db9_74fe_4a20_986e_321dab2426c7.slice/crio-a489321b66ca04cf3474ae17d3379ed9af9e720fd38b794243c98f1ac2091de8 WatchSource:0}: Error finding container a489321b66ca04cf3474ae17d3379ed9af9e720fd38b794243c98f1ac2091de8: Status 404 returned error can't find the container with id a489321b66ca04cf3474ae17d3379ed9af9e720fd38b794243c98f1ac2091de8 Jul 30 00:43:19.917483 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:19.917452 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 00:43:20.146463 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:20.146368 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs\") pod \"network-metrics-daemon-4r4tk\" (UID: \"4d304196-189e-44d8-ab5e-27ddbb8e18c2\") " pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:20.146639 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:20.146560 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:43:20.146708 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:20.146643 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs podName:4d304196-189e-44d8-ab5e-27ddbb8e18c2 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:21.146622295 +0000 UTC m=+3.947579380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs") pod "network-metrics-daemon-4r4tk" (UID: "4d304196-189e-44d8-ab5e-27ddbb8e18c2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:43:20.247107 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:20.247061 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:20.247324 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:20.247151 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjgm2\" (UniqueName: \"kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2\") pod \"network-check-target-z66tw\" (UID: \"0eb4b506-e95c-452e-a2f1-a0b683231704\") " pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:20.247324 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:20.247196 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 00:43:20.247324 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:20.247293 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret podName:962582df-c2b5-4b1f-990a-18b7a1c3ccfd nodeName:}" failed. No retries permitted until 2026-07-30 00:43:21.247263572 +0000 UTC m=+4.048220659 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret") pod "global-pull-secret-syncer-dj2qr" (UID: "962582df-c2b5-4b1f-990a-18b7a1c3ccfd") : object "kube-system"/"original-pull-secret" not registered Jul 30 00:43:20.247324 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:20.247314 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 00:43:20.247539 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:20.247340 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 00:43:20.247539 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:20.247356 2564 projected.go:196] Error preparing data for projected volume kube-api-access-vjgm2 for pod openshift-network-diagnostics/network-check-target-z66tw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:43:20.247539 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:20.247421 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2 podName:0eb4b506-e95c-452e-a2f1-a0b683231704 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:21.2474023 +0000 UTC m=+4.048359389 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vjgm2" (UniqueName: "kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2") pod "network-check-target-z66tw" (UID: "0eb4b506-e95c-452e-a2f1-a0b683231704") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:43:20.565815 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:20.565707 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 00:38:20 +0000 UTC" deadline="2028-02-19 00:59:14.341421193 +0000 UTC" Jul 30 00:43:20.565815 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:20.565757 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13656h15m53.775666972s" Jul 30 00:43:20.675365 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:20.675323 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7xg42" event={"ID":"61e4710e-5c0b-475d-80d4-db9f1cc78e27","Type":"ContainerStarted","Data":"dec2cbe15816559be20d730afecfada15e967e71de24bd0f9396b1671e27ec18"} Jul 30 00:43:20.676576 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:20.676543 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jv24z" event={"ID":"06214db9-74fe-4a20-986e-321dab2426c7","Type":"ContainerStarted","Data":"a489321b66ca04cf3474ae17d3379ed9af9e720fd38b794243c98f1ac2091de8"} Jul 30 00:43:20.677669 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:20.677645 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tvd2g" event={"ID":"1fca91ee-9cbd-4da1-b318-96ab379123e0","Type":"ContainerStarted","Data":"e3d2cb2a7bdada111b5af72950f681ceee7928c0d7f9cf655a561eeeb3cf0102"} Jul 30 00:43:20.678730 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:20.678709 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" event={"ID":"de873ad8-b46e-4dc3-b179-2368592e10e6","Type":"ContainerStarted","Data":"b3254c27de4f5268b66643782fc9c7272acd0f8660dd9a0eb0bd5af0f54e0ce6"} Jul 30 00:43:20.679756 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:20.679736 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjkjg" event={"ID":"0404fd97-1663-44e4-bf04-6a588423f8ed","Type":"ContainerStarted","Data":"3503638cf5bd4cb7298ed824a3c034508ee8ebffdfb3d197c1d90f20e112ff31"} Jul 30 00:43:20.680758 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:20.680737 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xqfqf" event={"ID":"aef50669-7ed3-432c-ad80-62254f6a5ac7","Type":"ContainerStarted","Data":"1eb9dd5c2a8c55c261214b37e7161888d53461a36e7a80e97f699861b060c148"} Jul 30 00:43:20.682011 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:20.681990 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" event={"ID":"34b16659-933b-4d38-86d9-f52957eb9128","Type":"ContainerStarted","Data":"cb984cf06640d03254f1db8cf4349aefb98af536ff60eb96360501eba822a5dc"} Jul 30 00:43:20.683145 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:20.683123 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mshnj" event={"ID":"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0","Type":"ContainerStarted","Data":"4fba1f9863a8de43c3b82bd10b8e892a5f17fb75f20aabb2eef3aa5547421d46"} Jul 30 00:43:20.684355 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:20.684331 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ds2cm" event={"ID":"84f859e4-75b2-4f4d-b6af-292c6b5fe003","Type":"ContainerStarted","Data":"44d742fdfbae5f4dc113d3c646767d9bf93f430cb08f6c9a371363360d8953ef"} Jul 30 00:43:20.919295 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:20.919209 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 00:43:21.154356 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:21.154311 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs\") pod \"network-metrics-daemon-4r4tk\" (UID: \"4d304196-189e-44d8-ab5e-27ddbb8e18c2\") " pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:21.154548 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:21.154509 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:43:21.154618 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:21.154575 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs podName:4d304196-189e-44d8-ab5e-27ddbb8e18c2 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:23.154556896 +0000 UTC m=+5.955514004 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs") pod "network-metrics-daemon-4r4tk" (UID: "4d304196-189e-44d8-ab5e-27ddbb8e18c2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:43:21.255018 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:21.254982 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjgm2\" (UniqueName: \"kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2\") pod \"network-check-target-z66tw\" (UID: \"0eb4b506-e95c-452e-a2f1-a0b683231704\") " pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:21.255136 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:21.255068 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:21.255203 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:21.255181 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 00:43:21.255203 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:21.255182 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 00:43:21.255312 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:21.255209 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 00:43:21.255312 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:21.255249 2564 projected.go:196] Error preparing data for projected volume kube-api-access-vjgm2 for pod openshift-network-diagnostics/network-check-target-z66tw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:43:21.255413 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:21.255250 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret podName:962582df-c2b5-4b1f-990a-18b7a1c3ccfd nodeName:}" failed. No retries permitted until 2026-07-30 00:43:23.255229366 +0000 UTC m=+6.056186454 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret") pod "global-pull-secret-syncer-dj2qr" (UID: "962582df-c2b5-4b1f-990a-18b7a1c3ccfd") : object "kube-system"/"original-pull-secret" not registered Jul 30 00:43:21.255413 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:21.255361 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2 podName:0eb4b506-e95c-452e-a2f1-a0b683231704 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:23.25534182 +0000 UTC m=+6.056298913 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vjgm2" (UniqueName: "kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2") pod "network-check-target-z66tw" (UID: "0eb4b506-e95c-452e-a2f1-a0b683231704") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:43:21.566362 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:21.565937 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 00:38:20 +0000 UTC" deadline="2028-01-11 22:21:47.718095683 +0000 UTC" Jul 30 00:43:21.566362 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:21.566335 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12741h38m26.151769242s" Jul 30 00:43:21.666349 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:21.665453 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:21.666349 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:21.665610 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:21.666349 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:21.665701 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:21.666349 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:21.665777 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:21.666349 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:21.665850 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:21.666349 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:21.665920 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:21.697564 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:21.697522 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-247.ec2.internal" event={"ID":"f42d3ae68cadbebcf1e1b0f116456906","Type":"ContainerStarted","Data":"eb9de0a40368299f7934431534d3a95d14511727264b8e37410bc77b35de502b"} Jul 30 00:43:22.705209 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:22.705170 2564 generic.go:356] "Generic (PLEG): container finished" podID="3394a8fe621d4462db0fc96b99ef5cf1" containerID="bd89754dd31794c90f88a423185aa61f101843147721ab925b85c1f6a66a98ee" exitCode=0 Jul 30 00:43:22.705722 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:22.705688 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal" event={"ID":"3394a8fe621d4462db0fc96b99ef5cf1","Type":"ContainerDied","Data":"bd89754dd31794c90f88a423185aa61f101843147721ab925b85c1f6a66a98ee"} Jul 30 00:43:22.722392 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:22.722327 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-247.ec2.internal" podStartSLOduration=3.722301918 podStartE2EDuration="3.722301918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:21.715843867 +0000 UTC m=+4.516800976" watchObservedRunningTime="2026-07-30 00:43:22.722301918 +0000 UTC m=+5.523259027" Jul 30 00:43:23.176086 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:23.175992 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs\") pod \"network-metrics-daemon-4r4tk\" (UID: \"4d304196-189e-44d8-ab5e-27ddbb8e18c2\") " pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:23.176267 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:23.176186 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:43:23.176267 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:23.176253 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs podName:4d304196-189e-44d8-ab5e-27ddbb8e18c2 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:27.176233783 +0000 UTC m=+9.977190871 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs") pod "network-metrics-daemon-4r4tk" (UID: "4d304196-189e-44d8-ab5e-27ddbb8e18c2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:43:23.278196 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:23.277162 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:23.278196 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:23.277286 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjgm2\" (UniqueName: \"kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2\") pod \"network-check-target-z66tw\" (UID: \"0eb4b506-e95c-452e-a2f1-a0b683231704\") " pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:23.278196 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:23.277466 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 00:43:23.278196 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:23.277492 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 00:43:23.278196 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:23.277508 2564 projected.go:196] Error preparing data for projected volume kube-api-access-vjgm2 for pod openshift-network-diagnostics/network-check-target-z66tw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:43:23.278196 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:23.277569 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2 podName:0eb4b506-e95c-452e-a2f1-a0b683231704 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:27.277549674 +0000 UTC m=+10.078506767 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vjgm2" (UniqueName: "kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2") pod "network-check-target-z66tw" (UID: "0eb4b506-e95c-452e-a2f1-a0b683231704") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:43:23.278196 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:23.278075 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 00:43:23.278196 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:23.278132 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret podName:962582df-c2b5-4b1f-990a-18b7a1c3ccfd nodeName:}" failed. No retries permitted until 2026-07-30 00:43:27.278115403 +0000 UTC m=+10.079072492 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret") pod "global-pull-secret-syncer-dj2qr" (UID: "962582df-c2b5-4b1f-990a-18b7a1c3ccfd") : object "kube-system"/"original-pull-secret" not registered Jul 30 00:43:23.663589 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:23.663084 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:23.663589 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:23.663101 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:23.663589 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:23.663234 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:23.663589 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:23.663278 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:23.663589 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:23.663357 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:23.663589 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:23.663438 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:25.662575 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:25.662457 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:25.662575 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:25.662501 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:25.663467 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:25.662609 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:25.663467 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:25.662457 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:25.663467 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:25.663063 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:25.663467 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:25.663100 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:27.213838 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:27.213794 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs\") pod \"network-metrics-daemon-4r4tk\" (UID: \"4d304196-189e-44d8-ab5e-27ddbb8e18c2\") " pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:27.214205 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:27.214020 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:43:27.214205 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:27.214085 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs podName:4d304196-189e-44d8-ab5e-27ddbb8e18c2 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:35.214065691 +0000 UTC m=+18.015022776 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs") pod "network-metrics-daemon-4r4tk" (UID: "4d304196-189e-44d8-ab5e-27ddbb8e18c2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:43:27.314418 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:27.314384 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:27.314578 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:27.314448 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjgm2\" (UniqueName: \"kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2\") pod \"network-check-target-z66tw\" (UID: \"0eb4b506-e95c-452e-a2f1-a0b683231704\") " pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:27.314660 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:27.314595 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 00:43:27.314660 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:27.314612 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 00:43:27.314660 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:27.314624 2564 projected.go:196] Error preparing data for projected volume kube-api-access-vjgm2 for pod openshift-network-diagnostics/network-check-target-z66tw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:43:27.314791 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:27.314678 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2 podName:0eb4b506-e95c-452e-a2f1-a0b683231704 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:35.31466172 +0000 UTC m=+18.115618819 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vjgm2" (UniqueName: "kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2") pod "network-check-target-z66tw" (UID: "0eb4b506-e95c-452e-a2f1-a0b683231704") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:43:27.315108 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:27.315086 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 00:43:27.315217 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:27.315143 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret podName:962582df-c2b5-4b1f-990a-18b7a1c3ccfd nodeName:}" failed. No retries permitted until 2026-07-30 00:43:35.315127775 +0000 UTC m=+18.116084860 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret") pod "global-pull-secret-syncer-dj2qr" (UID: "962582df-c2b5-4b1f-990a-18b7a1c3ccfd") : object "kube-system"/"original-pull-secret" not registered Jul 30 00:43:27.664139 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:27.663667 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:27.664139 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:27.663793 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:27.664139 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:27.663800 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:27.664139 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:27.663949 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:27.664139 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:27.663959 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:27.664139 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:27.664043 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:29.663707 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:29.663236 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:29.663707 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:29.663359 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:29.663707 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:29.663588 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:29.663707 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:29.663706 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:29.664359 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:29.663755 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:29.664359 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:29.663829 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:31.663400 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:31.663365 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:31.663850 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:31.663411 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:31.663850 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:31.663365 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:31.663850 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:31.663512 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:31.663850 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:31.663617 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:31.663850 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:31.663677 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:33.663224 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:33.663091 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:33.663224 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:33.663108 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:33.663224 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:33.663133 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:33.663224 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:33.663220 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:33.663852 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:33.663321 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:33.663852 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:33.663390 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:35.274448 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:35.274413 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs\") pod \"network-metrics-daemon-4r4tk\" (UID: \"4d304196-189e-44d8-ab5e-27ddbb8e18c2\") " pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:35.274839 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:35.274560 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:43:35.274839 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:35.274617 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs podName:4d304196-189e-44d8-ab5e-27ddbb8e18c2 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:51.274600924 +0000 UTC m=+34.075558013 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs") pod "network-metrics-daemon-4r4tk" (UID: "4d304196-189e-44d8-ab5e-27ddbb8e18c2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:43:35.375097 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:35.375063 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjgm2\" (UniqueName: \"kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2\") pod \"network-check-target-z66tw\" (UID: \"0eb4b506-e95c-452e-a2f1-a0b683231704\") " pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:35.375294 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:35.375127 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:35.375294 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:35.375225 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 00:43:35.375294 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:35.375278 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret podName:962582df-c2b5-4b1f-990a-18b7a1c3ccfd nodeName:}" failed. No retries permitted until 2026-07-30 00:43:51.375264623 +0000 UTC m=+34.176221711 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret") pod "global-pull-secret-syncer-dj2qr" (UID: "962582df-c2b5-4b1f-990a-18b7a1c3ccfd") : object "kube-system"/"original-pull-secret" not registered Jul 30 00:43:35.375400 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:35.375229 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 00:43:35.375400 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:35.375327 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 00:43:35.375400 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:35.375340 2564 projected.go:196] Error preparing data for projected volume kube-api-access-vjgm2 for pod openshift-network-diagnostics/network-check-target-z66tw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:43:35.375400 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:35.375383 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2 podName:0eb4b506-e95c-452e-a2f1-a0b683231704 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:51.37537138 +0000 UTC m=+34.176328479 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vjgm2" (UniqueName: "kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2") pod "network-check-target-z66tw" (UID: "0eb4b506-e95c-452e-a2f1-a0b683231704") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:43:35.663161 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:35.663121 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:35.663348 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:35.663128 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:35.663348 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:35.663250 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:35.663348 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:35.663134 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:35.663511 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:35.663341 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:35.663511 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:35.663404 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:37.663301 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:37.663262 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:37.663762 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:37.663369 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:37.663762 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:37.663369 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:37.663762 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:37.663452 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:37.663762 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:37.663480 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:37.663762 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:37.663543 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:39.662764 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:39.662728 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:39.662764 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:39.662748 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:39.663309 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:39.662736 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:39.663309 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:39.662846 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:39.663309 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:39.662930 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:39.663309 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:39.662999 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:40.751461 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.751094 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" event={"ID":"de873ad8-b46e-4dc3-b179-2368592e10e6","Type":"ContainerStarted","Data":"882bede14f6190a5605230dc86d669061ad4c53a5a8183023f36e32bb682bc42"} Jul 30 00:43:40.752516 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.752494 2564 generic.go:356] "Generic (PLEG): container finished" podID="0404fd97-1663-44e4-bf04-6a588423f8ed" containerID="c048096096a78850b16efb2429ca0baf51aa100db0bd0f20f7c419601b2ff51c" exitCode=0 Jul 30 00:43:40.752580 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.752564 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjkjg" event={"ID":"0404fd97-1663-44e4-bf04-6a588423f8ed","Type":"ContainerDied","Data":"c048096096a78850b16efb2429ca0baf51aa100db0bd0f20f7c419601b2ff51c"} Jul 30 00:43:40.753956 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.753935 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xqfqf" event={"ID":"aef50669-7ed3-432c-ad80-62254f6a5ac7","Type":"ContainerStarted","Data":"81ea77c777b82dd0750525dea0d1df3cd05eb1b4ea5334a5a208f556ab0523e1"} Jul 30 00:43:40.755633 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.755613 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal" event={"ID":"3394a8fe621d4462db0fc96b99ef5cf1","Type":"ContainerStarted","Data":"07296151a022f7c3fae6a3b6c690929e2192465c8f36baf894893b0a9b2ed292"} Jul 30 00:43:40.758457 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.758437 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" event={"ID":"34b16659-933b-4d38-86d9-f52957eb9128","Type":"ContainerStarted","Data":"ef8483c8c33979b314019e2c9c25c6ede0032195e1bbc8fd012aa7ed270898dc"} Jul 30 00:43:40.758546 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.758502 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" event={"ID":"34b16659-933b-4d38-86d9-f52957eb9128","Type":"ContainerStarted","Data":"36e4ac3cc4b959a76aa4af6bedc258506c6fe7aba634f77f2540987c88c69ae3"} Jul 30 00:43:40.758602 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.758550 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" event={"ID":"34b16659-933b-4d38-86d9-f52957eb9128","Type":"ContainerStarted","Data":"f025deaa4e139d7bf6bea37d251e9b30bca5989fc5ef54f94a4de509ec027414"} Jul 30 00:43:40.758643 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.758602 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" event={"ID":"34b16659-933b-4d38-86d9-f52957eb9128","Type":"ContainerStarted","Data":"37bf38fbe79aaead84102d324f0db4b96e5b2cf38d0fe5de953940d3417de3ab"} Jul 30 00:43:40.758691 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.758648 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" event={"ID":"34b16659-933b-4d38-86d9-f52957eb9128","Type":"ContainerStarted","Data":"80d922774220692d925cc418a38f63fed2d3601388b46008534ac6ed648cdbfa"} Jul 30 00:43:40.758691 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.758679 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" event={"ID":"34b16659-933b-4d38-86d9-f52957eb9128","Type":"ContainerStarted","Data":"1d65af47325bedfeb624d350ff952240ee99207617c4fcfe9b1b7e4d1ea1d2ea"} Jul 30 00:43:40.759725 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.759706 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mshnj" event={"ID":"db531aba-6f1e-4ab2-b7c3-3b4c2cf7b0a0","Type":"ContainerStarted","Data":"4aac576210bcad6b75822cb071d00819f0666d86e90b304deacbb533a4da6d1b"} Jul 30 00:43:40.760940 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.760918 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ds2cm" event={"ID":"84f859e4-75b2-4f4d-b6af-292c6b5fe003","Type":"ContainerStarted","Data":"53cad9fa74616ee8da82f42cd94dbffe11c8a15395ee97f9c97f75c52b444b39"} Jul 30 00:43:40.762108 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.762090 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7xg42" event={"ID":"61e4710e-5c0b-475d-80d4-db9f1cc78e27","Type":"ContainerStarted","Data":"065845e256dffdabbe24c8076b4b80c2a14fea69d514a222139526d03d9a9290"} Jul 30 00:43:40.763241 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.763223 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tvd2g" event={"ID":"1fca91ee-9cbd-4da1-b318-96ab379123e0","Type":"ContainerStarted","Data":"4e03148b2d4cf10e91d5cded2624e64acdaf8f20ceef751c25b3f1d955756a97"} Jul 30 00:43:40.790689 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.790631 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7xg42" podStartSLOduration=2.962488636 podStartE2EDuration="22.790614632s" totalImagesPullingTime="19.828125996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:40.790230848 +0000 UTC m=+23.591187955" watchObservedRunningTime="2026-07-30 00:43:40.790614632 +0000 UTC m=+23.591571738" Jul 30 00:43:40.803364 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.803322 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xqfqf" podStartSLOduration=7.44357418 podStartE2EDuration="22.803306975s" totalImagesPullingTime="15.359732795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:40.802667804 +0000 UTC m=+23.603624910" watchObservedRunningTime="2026-07-30 00:43:40.803306975 +0000 UTC m=+23.604264117" Jul 30 00:43:40.815487 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.815442 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ds2cm" podStartSLOduration=7.451705138 podStartE2EDuration="22.815428872s" totalImagesPullingTime="15.363723734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:40.815215041 +0000 UTC m=+23.616172147" watchObservedRunningTime="2026-07-30 00:43:40.815428872 +0000 UTC m=+23.616385978" Jul 30 00:43:40.829751 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.829694 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mshnj" podStartSLOduration=2.994371599 podStartE2EDuration="22.82968199s" totalImagesPullingTime="19.835310391s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:40.829267326 +0000 UTC m=+23.630224432" watchObservedRunningTime="2026-07-30 00:43:40.82968199 +0000 UTC m=+23.630639095" Jul 30 00:43:40.846751 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.846697 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tvd2g" podStartSLOduration=3.021468697 podStartE2EDuration="22.846681462s" totalImagesPullingTime="19.825212765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:40.846130274 +0000 UTC m=+23.647087380" watchObservedRunningTime="2026-07-30 00:43:40.846681462 +0000 UTC m=+23.647638568" Jul 30 00:43:40.865532 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:40.865480 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-247.ec2.internal" podStartSLOduration=21.86546377 podStartE2EDuration="21.86546377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:40.864997483 +0000 UTC m=+23.665954616" watchObservedRunningTime="2026-07-30 00:43:40.86546377 +0000 UTC m=+23.666420876" Jul 30 00:43:41.019409 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:41.019380 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 30 00:43:41.599891 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:41.599718 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T00:43:41.019399229Z","UUID":"fd4596b4-9776-401c-a371-a5b43be4ab06","Handler":null,"Name":"","Endpoint":""} Jul 30 00:43:41.601490 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:41.601470 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 30 00:43:41.601618 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:41.601513 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 30 00:43:41.666857 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:41.666780 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:41.667043 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:41.666885 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:41.667043 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:41.666913 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:41.667155 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:41.667033 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:41.667155 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:41.667066 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:41.667155 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:41.667141 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:41.767649 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:41.767601 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jv24z" event={"ID":"06214db9-74fe-4a20-986e-321dab2426c7","Type":"ContainerStarted","Data":"b49f672ce2726500be4c8a46cac6d76eb90adb7cf6ee00983992b0ddb52aa572"} Jul 30 00:43:41.769821 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:41.769753 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" event={"ID":"de873ad8-b46e-4dc3-b179-2368592e10e6","Type":"ContainerStarted","Data":"4fca40ade57ea950611ab68460e706e88551955c09baa84cac00650a55c6f035"} Jul 30 00:43:41.781432 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:41.781373 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jv24z" podStartSLOduration=4.01743187 podStartE2EDuration="23.781352356s" totalImagesPullingTime="19.763920486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:41.780665686 +0000 UTC m=+24.581622791" watchObservedRunningTime="2026-07-30 00:43:41.781352356 +0000 UTC m=+24.582309462" Jul 30 00:43:42.776676 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:42.776581 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" event={"ID":"34b16659-933b-4d38-86d9-f52957eb9128","Type":"ContainerStarted","Data":"ce25b2ee2b6ab71b6ae80d17ec2cf3d81914af95a345b93483dd4db7f1b7469e"} Jul 30 00:43:43.663036 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:43.662808 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:43.663352 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:43.662808 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:43.663352 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:43.663141 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:43.663352 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:43.663213 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:43.663352 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:43.662809 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:43.663352 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:43.663297 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:43.780766 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:43.780731 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" event={"ID":"de873ad8-b46e-4dc3-b179-2368592e10e6","Type":"ContainerStarted","Data":"6ff0acddef3a12834d3b7395fd8e97a53679a1c22ec5fa9a60415091310de29c"} Jul 30 00:43:43.797125 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:43.797075 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z55qt" podStartSLOduration=3.283146915 podStartE2EDuration="25.797057498s" totalImagesPullingTime="22.513910583s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:43.795652663 +0000 UTC m=+26.596609770" watchObservedRunningTime="2026-07-30 00:43:43.797057498 +0000 UTC m=+26.598014603" Jul 30 00:43:44.601056 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:44.601016 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ds2cm" Jul 30 00:43:44.601728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:44.601706 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ds2cm" Jul 30 00:43:44.925600 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:44.925518 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ds2cm" Jul 30 00:43:44.926227 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:44.926201 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ds2cm" Jul 30 00:43:45.662757 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:45.662583 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:45.662879 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:45.662623 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:45.662879 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:45.662864 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:45.663020 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:45.662624 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:45.663020 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:45.662972 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:45.663122 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:45.663078 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:45.788698 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:45.788602 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" event={"ID":"34b16659-933b-4d38-86d9-f52957eb9128","Type":"ContainerStarted","Data":"8b11ad8448357ada17087c1e34f124ae67e44828a751ff5e2bb19f69874e6ce8"} Jul 30 00:43:45.788862 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:45.788839 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:45.788947 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:45.788931 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:45.790479 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:45.790455 2564 generic.go:356] "Generic (PLEG): container finished" podID="0404fd97-1663-44e4-bf04-6a588423f8ed" containerID="807ccfd08010a9918e72d4934294d8aff2f58a4a8c55d5d2be51bb457448fe8a" exitCode=0 Jul 30 00:43:45.790617 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:45.790482 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjkjg" event={"ID":"0404fd97-1663-44e4-bf04-6a588423f8ed","Type":"ContainerDied","Data":"807ccfd08010a9918e72d4934294d8aff2f58a4a8c55d5d2be51bb457448fe8a"} Jul 30 00:43:45.804721 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:45.804697 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:45.814688 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:45.814649 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" podStartSLOduration=7.970755399 podStartE2EDuration="27.814635565s" totalImagesPullingTime="19.843880166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:45.813092319 +0000 UTC m=+28.614049424" watchObservedRunningTime="2026-07-30 00:43:45.814635565 +0000 UTC m=+28.615592670" Jul 30 00:43:46.793297 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:46.793268 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:46.808318 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:46.808287 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:43:47.120041 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:47.120009 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dj2qr"] Jul 30 00:43:47.120194 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:47.120151 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:47.120285 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:47.120262 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:47.123088 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:47.123051 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z66tw"] Jul 30 00:43:47.123229 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:47.123164 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:47.123294 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:47.123260 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:47.123598 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:47.123574 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4r4tk"] Jul 30 00:43:47.123683 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:47.123666 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:47.123767 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:47.123749 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:47.796691 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:47.796662 2564 generic.go:356] "Generic (PLEG): container finished" podID="0404fd97-1663-44e4-bf04-6a588423f8ed" containerID="884d8349be693eb6bd9b2b251389bb4808c91e195ee9a710221e0c2541f93d54" exitCode=0 Jul 30 00:43:47.797098 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:47.796750 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjkjg" event={"ID":"0404fd97-1663-44e4-bf04-6a588423f8ed","Type":"ContainerDied","Data":"884d8349be693eb6bd9b2b251389bb4808c91e195ee9a710221e0c2541f93d54"} Jul 30 00:43:48.662600 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:48.662573 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:48.662600 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:48.662600 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:48.662850 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:48.662574 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:48.662850 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:48.662691 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:48.662850 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:48.662796 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:48.663018 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:48.662874 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:49.803093 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:49.802890 2564 generic.go:356] "Generic (PLEG): container finished" podID="0404fd97-1663-44e4-bf04-6a588423f8ed" containerID="a540a1dc17825191431087714d213ed1f93dbc5cf58a1f947d7075de22cc8621" exitCode=0 Jul 30 00:43:49.803701 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:49.802930 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjkjg" event={"ID":"0404fd97-1663-44e4-bf04-6a588423f8ed","Type":"ContainerDied","Data":"a540a1dc17825191431087714d213ed1f93dbc5cf58a1f947d7075de22cc8621"} Jul 30 00:43:50.662939 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:50.662884 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:50.663134 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:50.662905 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:50.663134 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:50.663030 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z66tw" podUID="0eb4b506-e95c-452e-a2f1-a0b683231704" Jul 30 00:43:50.663134 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:50.663125 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj2qr" podUID="962582df-c2b5-4b1f-990a-18b7a1c3ccfd" Jul 30 00:43:50.663308 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:50.662909 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:50.663308 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:50.663238 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r4tk" podUID="4d304196-189e-44d8-ab5e-27ddbb8e18c2" Jul 30 00:43:51.288397 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:51.288318 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs\") pod \"network-metrics-daemon-4r4tk\" (UID: \"4d304196-189e-44d8-ab5e-27ddbb8e18c2\") " pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:51.288803 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:51.288448 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:43:51.288803 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:51.288521 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs podName:4d304196-189e-44d8-ab5e-27ddbb8e18c2 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:23.288499661 +0000 UTC m=+66.089456750 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs") pod "network-metrics-daemon-4r4tk" (UID: "4d304196-189e-44d8-ab5e-27ddbb8e18c2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:43:51.389348 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:51.389306 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:51.389530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:51.389375 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjgm2\" (UniqueName: \"kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2\") pod \"network-check-target-z66tw\" (UID: \"0eb4b506-e95c-452e-a2f1-a0b683231704\") " pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:51.389530 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:51.389488 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 00:43:51.389633 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:51.389574 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret podName:962582df-c2b5-4b1f-990a-18b7a1c3ccfd nodeName:}" failed. No retries permitted until 2026-07-30 00:44:23.389554082 +0000 UTC m=+66.190511181 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret") pod "global-pull-secret-syncer-dj2qr" (UID: "962582df-c2b5-4b1f-990a-18b7a1c3ccfd") : object "kube-system"/"original-pull-secret" not registered Jul 30 00:43:51.389633 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:51.389495 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 00:43:51.389633 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:51.389611 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 00:43:51.389633 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:51.389624 2564 projected.go:196] Error preparing data for projected volume kube-api-access-vjgm2 for pod openshift-network-diagnostics/network-check-target-z66tw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:43:51.389843 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:51.389682 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2 podName:0eb4b506-e95c-452e-a2f1-a0b683231704 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:23.389667311 +0000 UTC m=+66.190624399 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vjgm2" (UniqueName: "kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2") pod "network-check-target-z66tw" (UID: "0eb4b506-e95c-452e-a2f1-a0b683231704") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:43:52.527780 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.527693 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-247.ec2.internal" event="NodeReady" Jul 30 00:43:52.528324 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.527850 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 30 00:43:52.562829 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.562795 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56b77fd5b4-2zm7w"] Jul 30 00:43:52.567893 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.567871 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.570532 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.570508 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 30 00:43:52.570668 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.570541 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zt6vm\"" Jul 30 00:43:52.570802 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.570785 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 30 00:43:52.570867 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.570784 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 30 00:43:52.575248 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.575227 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 30 00:43:52.577535 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.577512 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8cxk4"] Jul 30 00:43:52.581019 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.581003 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8cxk4" Jul 30 00:43:52.583420 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.583398 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56b77fd5b4-2zm7w"] Jul 30 00:43:52.583957 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.583766 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r2q5g\"" Jul 30 00:43:52.583957 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.583795 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 30 00:43:52.583957 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.583877 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 30 00:43:52.586723 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.586549 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4t9fb"] Jul 30 00:43:52.590189 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.590167 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4t9fb" Jul 30 00:43:52.591007 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.590988 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8cxk4"] Jul 30 00:43:52.592854 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.592608 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mgwl2\"" Jul 30 00:43:52.592854 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.592648 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 30 00:43:52.592854 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.592684 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 30 00:43:52.592854 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.592779 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 30 00:43:52.600284 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.600256 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4t9fb"] Jul 30 00:43:52.663434 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.663400 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:43:52.663434 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.663439 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:43:52.663660 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.663456 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:43:52.666200 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.666171 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 30 00:43:52.666529 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.666509 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 00:43:52.666652 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.666553 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 30 00:43:52.666741 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.666682 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vdmkk\"" Jul 30 00:43:52.666741 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.666727 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r24dv\"" Jul 30 00:43:52.666847 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.666784 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 00:43:52.699716 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.699674 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:43:52.699716 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.699715 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftzfx\" (UniqueName: \"kubernetes.io/projected/7d1f5f34-14d3-4458-9c22-f6b33bcea80e-kube-api-access-ftzfx\") pod \"ingress-canary-4t9fb\" (UID: \"7d1f5f34-14d3-4458-9c22-f6b33bcea80e\") " pod="openshift-ingress-canary/ingress-canary-4t9fb" Jul 30 00:43:52.699937 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.699819 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-config-volume\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:43:52.699937 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.699849 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-tmp-dir\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:43:52.699937 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.699886 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6mr2\" (UniqueName: \"kubernetes.io/projected/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-kube-api-access-g6mr2\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:43:52.700071 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.699951 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-image-registry-private-configuration\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.700071 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.699971 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-ca-trust-extracted\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.700071 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.699990 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-installation-pull-secrets\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.700071 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.700030 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-certificates\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.700071 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.700055 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-bound-sa-token\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.700266 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.700101 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d1f5f34-14d3-4458-9c22-f6b33bcea80e-cert\") pod \"ingress-canary-4t9fb\" (UID: \"7d1f5f34-14d3-4458-9c22-f6b33bcea80e\") " pod="openshift-ingress-canary/ingress-canary-4t9fb" Jul 30 00:43:52.700266 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.700131 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.700266 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.700155 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9pg5\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-kube-api-access-f9pg5\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.700266 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.700183 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-trusted-ca\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.801065 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.800977 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-config-volume\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:43:52.801065 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801013 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-tmp-dir\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:43:52.801065 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801054 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6mr2\" (UniqueName: \"kubernetes.io/projected/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-kube-api-access-g6mr2\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:43:52.801318 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801081 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-image-registry-private-configuration\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.801318 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801108 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-ca-trust-extracted\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.801318 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801137 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-installation-pull-secrets\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.801318 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801177 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-certificates\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.801318 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801200 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-bound-sa-token\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.801318 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801224 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d1f5f34-14d3-4458-9c22-f6b33bcea80e-cert\") pod \"ingress-canary-4t9fb\" (UID: \"7d1f5f34-14d3-4458-9c22-f6b33bcea80e\") " pod="openshift-ingress-canary/ingress-canary-4t9fb" Jul 30 00:43:52.801318 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801251 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.801318 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801277 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9pg5\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-kube-api-access-f9pg5\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.801318 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801313 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-trusted-ca\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.801711 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801380 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:43:52.801711 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801420 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftzfx\" (UniqueName: \"kubernetes.io/projected/7d1f5f34-14d3-4458-9c22-f6b33bcea80e-kube-api-access-ftzfx\") pod \"ingress-canary-4t9fb\" (UID: \"7d1f5f34-14d3-4458-9c22-f6b33bcea80e\") " pod="openshift-ingress-canary/ingress-canary-4t9fb" Jul 30 00:43:52.801711 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801415 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-tmp-dir\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:43:52.801711 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:52.801548 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 00:43:52.801711 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:52.801623 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls podName:5c59f8b2-e65b-4f30-93bb-16a06e3d5f45 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:53.301600166 +0000 UTC m=+36.102557262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls") pod "dns-default-8cxk4" (UID: "5c59f8b2-e65b-4f30-93bb-16a06e3d5f45") : secret "dns-default-metrics-tls" not found Jul 30 00:43:52.801961 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801759 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-ca-trust-extracted\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.802024 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.801963 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-certificates\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.802078 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:52.802069 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 00:43:52.802125 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:52.802084 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56b77fd5b4-2zm7w: secret "image-registry-tls" not found Jul 30 00:43:52.802177 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:52.802139 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls podName:4af9a9e9-f2a5-4df9-aea5-7439becee5e1 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:53.302115371 +0000 UTC m=+36.103072459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls") pod "image-registry-56b77fd5b4-2zm7w" (UID: "4af9a9e9-f2a5-4df9-aea5-7439becee5e1") : secret "image-registry-tls" not found Jul 30 00:43:52.802230 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.802193 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-config-volume\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:43:52.802910 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.802884 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-trusted-ca\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.805918 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.805734 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-installation-pull-secrets\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.805918 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.805836 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-image-registry-private-configuration\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.806079 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.805989 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d1f5f34-14d3-4458-9c22-f6b33bcea80e-cert\") pod \"ingress-canary-4t9fb\" (UID: \"7d1f5f34-14d3-4458-9c22-f6b33bcea80e\") " pod="openshift-ingress-canary/ingress-canary-4t9fb" Jul 30 00:43:52.811110 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.811086 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftzfx\" (UniqueName: \"kubernetes.io/projected/7d1f5f34-14d3-4458-9c22-f6b33bcea80e-kube-api-access-ftzfx\") pod \"ingress-canary-4t9fb\" (UID: \"7d1f5f34-14d3-4458-9c22-f6b33bcea80e\") " pod="openshift-ingress-canary/ingress-canary-4t9fb" Jul 30 00:43:52.811216 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.811111 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-bound-sa-token\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.811571 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.811544 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6mr2\" (UniqueName: \"kubernetes.io/projected/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-kube-api-access-g6mr2\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:43:52.811654 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.811623 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9pg5\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-kube-api-access-f9pg5\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:52.901408 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:52.901366 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4t9fb" Jul 30 00:43:53.049061 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:53.049011 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:53.128324 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:53.128285 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:53.305968 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:53.305928 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:53.306158 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:53.305988 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:43:53.306158 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:53.306085 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 00:43:53.306158 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:53.306106 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56b77fd5b4-2zm7w: secret "image-registry-tls" not found Jul 30 00:43:53.306158 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:53.306128 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 00:43:53.306355 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:53.306174 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls podName:4af9a9e9-f2a5-4df9-aea5-7439becee5e1 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:54.306153822 +0000 UTC m=+37.107110907 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls") pod "image-registry-56b77fd5b4-2zm7w" (UID: "4af9a9e9-f2a5-4df9-aea5-7439becee5e1") : secret "image-registry-tls" not found Jul 30 00:43:53.306355 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:53.306193 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls podName:5c59f8b2-e65b-4f30-93bb-16a06e3d5f45 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:54.306183914 +0000 UTC m=+37.107141005 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls") pod "dns-default-8cxk4" (UID: "5c59f8b2-e65b-4f30-93bb-16a06e3d5f45") : secret "dns-default-metrics-tls" not found Jul 30 00:43:54.004334 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:54.004142 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4t9fb"] Jul 30 00:43:54.008186 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:43:54.008156 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d1f5f34_14d3_4458_9c22_f6b33bcea80e.slice/crio-bbf3ac735226437a41a65d3f064ab9f29b48831665342b537b988ad27faba4bd WatchSource:0}: Error finding container bbf3ac735226437a41a65d3f064ab9f29b48831665342b537b988ad27faba4bd: Status 404 returned error can't find the container with id bbf3ac735226437a41a65d3f064ab9f29b48831665342b537b988ad27faba4bd Jul 30 00:43:54.314446 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:54.314407 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:54.314446 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:54.314454 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:43:54.314647 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:54.314570 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 00:43:54.314647 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:54.314595 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56b77fd5b4-2zm7w: secret "image-registry-tls" not found Jul 30 00:43:54.314647 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:54.314599 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 00:43:54.314732 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:54.314678 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls podName:5c59f8b2-e65b-4f30-93bb-16a06e3d5f45 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:56.314639749 +0000 UTC m=+39.115596853 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls") pod "dns-default-8cxk4" (UID: "5c59f8b2-e65b-4f30-93bb-16a06e3d5f45") : secret "dns-default-metrics-tls" not found Jul 30 00:43:54.314732 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:54.314699 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls podName:4af9a9e9-f2a5-4df9-aea5-7439becee5e1 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:56.314689836 +0000 UTC m=+39.115646919 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls") pod "image-registry-56b77fd5b4-2zm7w" (UID: "4af9a9e9-f2a5-4df9-aea5-7439becee5e1") : secret "image-registry-tls" not found Jul 30 00:43:54.668363 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:54.668311 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:54.735274 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:54.735239 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:54.816461 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:54.816426 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4t9fb" event={"ID":"7d1f5f34-14d3-4458-9c22-f6b33bcea80e","Type":"ContainerStarted","Data":"bbf3ac735226437a41a65d3f064ab9f29b48831665342b537b988ad27faba4bd"} Jul 30 00:43:54.819181 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:54.819156 2564 generic.go:356] "Generic (PLEG): container finished" podID="0404fd97-1663-44e4-bf04-6a588423f8ed" containerID="e33ee532d3af0d70549bcd5dce2b55d05cb8416943f41cb2cf233f37eed80387" exitCode=0 Jul 30 00:43:54.819313 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:54.819204 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjkjg" event={"ID":"0404fd97-1663-44e4-bf04-6a588423f8ed","Type":"ContainerDied","Data":"e33ee532d3af0d70549bcd5dce2b55d05cb8416943f41cb2cf233f37eed80387"} Jul 30 00:43:54.998806 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:54.998761 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:55.099367 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:55.099333 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:55.668603 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:55.668567 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:55.764025 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:55.763991 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:55.824958 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:55.824925 2564 generic.go:356] "Generic (PLEG): container finished" podID="0404fd97-1663-44e4-bf04-6a588423f8ed" containerID="59f74664575f32717a8bb38c56188cfb8daa0d1146e07f7538223d179812761c" exitCode=0 Jul 30 00:43:55.825132 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:55.824968 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjkjg" event={"ID":"0404fd97-1663-44e4-bf04-6a588423f8ed","Type":"ContainerDied","Data":"59f74664575f32717a8bb38c56188cfb8daa0d1146e07f7538223d179812761c"} Jul 30 00:43:55.854771 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:55.854650 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:56.331472 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:56.331434 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:43:56.331934 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:56.331551 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:43:56.331934 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:56.331610 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 00:43:56.331934 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:56.331652 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 00:43:56.331934 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:56.331665 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56b77fd5b4-2zm7w: secret "image-registry-tls" not found Jul 30 00:43:56.331934 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:56.331683 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls podName:5c59f8b2-e65b-4f30-93bb-16a06e3d5f45 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:00.331664992 +0000 UTC m=+43.132622093 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls") pod "dns-default-8cxk4" (UID: "5c59f8b2-e65b-4f30-93bb-16a06e3d5f45") : secret "dns-default-metrics-tls" not found Jul 30 00:43:56.331934 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:43:56.331700 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls podName:4af9a9e9-f2a5-4df9-aea5-7439becee5e1 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:00.331690378 +0000 UTC m=+43.132647461 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls") pod "image-registry-56b77fd5b4-2zm7w" (UID: "4af9a9e9-f2a5-4df9-aea5-7439becee5e1") : secret "image-registry-tls" not found Jul 30 00:43:56.694181 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:56.693925 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:56.755878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:56.755843 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:56.829847 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:56.829820 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjkjg" event={"ID":"0404fd97-1663-44e4-bf04-6a588423f8ed","Type":"ContainerStarted","Data":"1623ea2f602d02fdd5e5734093713765dbde2aa03b3e890ffb7f3c160f12a844"} Jul 30 00:43:56.854546 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:56.854489 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pjkjg" podStartSLOduration=8.186713303 podStartE2EDuration="38.854472582s" totalImagesPullingTime="30.667759279s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:56.852721187 +0000 UTC m=+39.653678292" watchObservedRunningTime="2026-07-30 00:43:56.854472582 +0000 UTC m=+39.655429688" Jul 30 00:43:56.961690 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:56.961655 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:57.062734 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:57.062699 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:57.195300 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:57.195207 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:57.598135 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:57.598102 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:57.664609 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:57.664570 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:57.833045 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:57.833007 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4t9fb" event={"ID":"7d1f5f34-14d3-4458-9c22-f6b33bcea80e","Type":"ContainerStarted","Data":"afa73fb6d2caed55c9651c6ddfafd9eb3692ad5db25c265bd8fa0e72ca6500ca"} Jul 30 00:43:57.849949 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:57.849851 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4t9fb" podStartSLOduration=2.920041451 podStartE2EDuration="5.849834986s" totalImagesPullingTime="2.929793535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:43:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:57.848998884 +0000 UTC m=+40.649955992" watchObservedRunningTime="2026-07-30 00:43:57.849834986 +0000 UTC m=+40.650792091" Jul 30 00:43:58.392389 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:58.392356 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:58.443801 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:58.443772 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:58.631243 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:58.631191 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:58.793943 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:58.793823 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:43:58.846009 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:43:58.845943 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:00.360977 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:00.360943 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:44:00.361345 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:00.360991 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:44:00.361345 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:00.361104 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 00:44:00.361345 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:00.361120 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 00:44:00.361345 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:00.361123 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56b77fd5b4-2zm7w: secret "image-registry-tls" not found Jul 30 00:44:00.361345 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:00.361178 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls podName:5c59f8b2-e65b-4f30-93bb-16a06e3d5f45 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:08.361163428 +0000 UTC m=+51.162120513 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls") pod "dns-default-8cxk4" (UID: "5c59f8b2-e65b-4f30-93bb-16a06e3d5f45") : secret "dns-default-metrics-tls" not found Jul 30 00:44:00.361345 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:00.361191 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls podName:4af9a9e9-f2a5-4df9-aea5-7439becee5e1 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:08.361185155 +0000 UTC m=+51.162142239 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls") pod "image-registry-56b77fd5b4-2zm7w" (UID: "4af9a9e9-f2a5-4df9-aea5-7439becee5e1") : secret "image-registry-tls" not found Jul 30 00:44:04.182609 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:04.182554 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:07.340979 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:07.340947 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:08.418681 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:08.418642 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:44:08.418681 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:08.418687 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:44:08.419171 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:08.418799 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 00:44:08.419171 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:08.418852 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls podName:5c59f8b2-e65b-4f30-93bb-16a06e3d5f45 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:24.418839098 +0000 UTC m=+67.219796182 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls") pod "dns-default-8cxk4" (UID: "5c59f8b2-e65b-4f30-93bb-16a06e3d5f45") : secret "dns-default-metrics-tls" not found Jul 30 00:44:08.419171 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:08.418797 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 00:44:08.419171 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:08.418916 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56b77fd5b4-2zm7w: secret "image-registry-tls" not found Jul 30 00:44:08.419171 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:08.418968 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls podName:4af9a9e9-f2a5-4df9-aea5-7439becee5e1 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:24.418954172 +0000 UTC m=+67.219911260 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls") pod "image-registry-56b77fd5b4-2zm7w" (UID: "4af9a9e9-f2a5-4df9-aea5-7439becee5e1") : secret "image-registry-tls" not found Jul 30 00:44:08.710450 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:08.710352 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:08.776724 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:08.776691 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:11.784143 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:11.784108 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:11.835668 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:11.835634 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:12.132541 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:12.132515 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:13.004047 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.004009 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:13.060792 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.060756 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:13.297757 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.297685 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-67c4d7bd47-tszsf"] Jul 30 00:44:13.299611 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.299597 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:13.303427 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.303399 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 30 00:44:13.303624 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.303610 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 30 00:44:13.303786 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.303772 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 30 00:44:13.304424 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.304409 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 30 00:44:13.304516 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.304501 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 30 00:44:13.304571 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.304502 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5k5jm\"" Jul 30 00:44:13.304844 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.304831 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 30 00:44:13.319848 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.319823 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-67c4d7bd47-tszsf"] Jul 30 00:44:13.458867 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.458825 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:13.459063 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.458920 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-stats-auth\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:13.459063 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.458947 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-default-certificate\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:13.459063 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.458965 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs7hl\" (UniqueName: \"kubernetes.io/projected/49165c64-0239-4c5d-b252-218818d09280-kube-api-access-zs7hl\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:13.459063 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.459003 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:13.464760 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.464728 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:13.560103 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.560017 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-stats-auth\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:13.560103 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.560053 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-default-certificate\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:13.560103 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.560074 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs7hl\" (UniqueName: \"kubernetes.io/projected/49165c64-0239-4c5d-b252-218818d09280-kube-api-access-zs7hl\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:13.560103 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.560103 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:13.560408 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.560188 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:13.560408 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:13.560279 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 00:44:13.560408 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:13.560334 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle podName:49165c64-0239-4c5d-b252-218818d09280 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:14.060307933 +0000 UTC m=+56.861265039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle") pod "router-default-67c4d7bd47-tszsf" (UID: "49165c64-0239-4c5d-b252-218818d09280") : configmap references non-existent config key: service-ca.crt Jul 30 00:44:13.560408 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:13.560361 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs podName:49165c64-0239-4c5d-b252-218818d09280 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:14.060350532 +0000 UTC m=+56.861307616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs") pod "router-default-67c4d7bd47-tszsf" (UID: "49165c64-0239-4c5d-b252-218818d09280") : secret "router-metrics-certs-default" not found Jul 30 00:44:13.564007 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.563986 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-stats-auth\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:13.564007 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.564002 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-default-certificate\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:13.568359 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.568317 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:13.571917 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.571854 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs7hl\" (UniqueName: \"kubernetes.io/projected/49165c64-0239-4c5d-b252-218818d09280-kube-api-access-zs7hl\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:13.861635 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.861600 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:13.910691 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:13.910652 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:14.064735 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:14.064687 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:14.065154 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:14.064768 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:14.065154 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:14.064840 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 00:44:14.065154 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:14.064913 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle podName:49165c64-0239-4c5d-b252-218818d09280 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:15.06488346 +0000 UTC m=+57.865840544 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle") pod "router-default-67c4d7bd47-tszsf" (UID: "49165c64-0239-4c5d-b252-218818d09280") : configmap references non-existent config key: service-ca.crt Jul 30 00:44:14.065154 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:14.064932 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs podName:49165c64-0239-4c5d-b252-218818d09280 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:15.064925256 +0000 UTC m=+57.865882339 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs") pod "router-default-67c4d7bd47-tszsf" (UID: "49165c64-0239-4c5d-b252-218818d09280") : secret "router-metrics-certs-default" not found Jul 30 00:44:14.244251 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:14.244172 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:14.345569 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:14.345541 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:15.073596 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:15.073562 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:15.074019 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:15.073627 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:15.074019 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:15.073745 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 00:44:15.074019 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:15.073826 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle podName:49165c64-0239-4c5d-b252-218818d09280 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:17.07380915 +0000 UTC m=+59.874766234 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle") pod "router-default-67c4d7bd47-tszsf" (UID: "49165c64-0239-4c5d-b252-218818d09280") : configmap references non-existent config key: service-ca.crt Jul 30 00:44:15.074019 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:15.073858 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs podName:49165c64-0239-4c5d-b252-218818d09280 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:17.07384228 +0000 UTC m=+59.874799364 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs") pod "router-default-67c4d7bd47-tszsf" (UID: "49165c64-0239-4c5d-b252-218818d09280") : secret "router-metrics-certs-default" not found Jul 30 00:44:17.087120 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:17.087072 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:17.087559 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:17.087165 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:17.087559 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:17.087259 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle podName:49165c64-0239-4c5d-b252-218818d09280 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:21.087234029 +0000 UTC m=+63.888191113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle") pod "router-default-67c4d7bd47-tszsf" (UID: "49165c64-0239-4c5d-b252-218818d09280") : configmap references non-existent config key: service-ca.crt Jul 30 00:44:17.087559 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:17.087262 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 00:44:17.087559 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:17.087312 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs podName:49165c64-0239-4c5d-b252-218818d09280 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:21.087301966 +0000 UTC m=+63.888259053 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs") pod "router-default-67c4d7bd47-tszsf" (UID: "49165c64-0239-4c5d-b252-218818d09280") : secret "router-metrics-certs-default" not found Jul 30 00:44:17.102014 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:17.101988 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:17.166719 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:17.166692 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:18.641979 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:18.641943 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:18.705041 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:18.705003 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:18.760577 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:18.760549 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:18.811595 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:18.811565 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kfmfq" Jul 30 00:44:19.305234 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:19.305196 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:19.405073 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:19.405029 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:21.057211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.057175 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:21.116210 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.116170 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:21.116374 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:21.116335 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle podName:49165c64-0239-4c5d-b252-218818d09280 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:29.116316707 +0000 UTC m=+71.917273792 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle") pod "router-default-67c4d7bd47-tszsf" (UID: "49165c64-0239-4c5d-b252-218818d09280") : configmap references non-existent config key: service-ca.crt Jul 30 00:44:21.116374 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.116363 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:21.116480 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:21.116468 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 00:44:21.116520 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:21.116505 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs podName:49165c64-0239-4c5d-b252-218818d09280 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:29.116496605 +0000 UTC m=+71.917453688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs") pod "router-default-67c4d7bd47-tszsf" (UID: "49165c64-0239-4c5d-b252-218818d09280") : secret "router-metrics-certs-default" not found Jul 30 00:44:21.242545 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.242507 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:21.314763 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.314681 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bg6hz"] Jul 30 00:44:21.319014 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.318987 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:21.321409 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.321388 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p9rd8\"" Jul 30 00:44:21.322063 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.322049 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 30 00:44:21.322156 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.322115 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 30 00:44:21.322156 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.322128 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 30 00:44:21.322156 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.322115 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 30 00:44:21.328514 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.328492 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bg6hz"] Jul 30 00:44:21.349511 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.349484 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:21.418176 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.418143 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/610a5275-fbb6-4501-bc53-a31f61798db0-crio-socket\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:21.418176 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.418179 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:21.418431 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.418298 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/610a5275-fbb6-4501-bc53-a31f61798db0-data-volume\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:21.418431 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.418335 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:21.418431 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.418358 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq8lx\" (UniqueName: \"kubernetes.io/projected/610a5275-fbb6-4501-bc53-a31f61798db0-kube-api-access-fq8lx\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:21.491011 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.490978 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:21.519310 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.519282 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/610a5275-fbb6-4501-bc53-a31f61798db0-data-volume\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:21.519423 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.519314 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:21.519423 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.519335 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq8lx\" (UniqueName: \"kubernetes.io/projected/610a5275-fbb6-4501-bc53-a31f61798db0-kube-api-access-fq8lx\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:21.519536 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:21.519427 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 00:44:21.519536 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:21.519498 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm podName:610a5275-fbb6-4501-bc53-a31f61798db0 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:22.019476604 +0000 UTC m=+64.820433702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bg6hz" (UID: "610a5275-fbb6-4501-bc53-a31f61798db0") : configmap "kube-rbac-proxy" not found Jul 30 00:44:21.519633 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.519596 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/610a5275-fbb6-4501-bc53-a31f61798db0-crio-socket\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:21.519633 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.519622 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:21.519708 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.519681 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/610a5275-fbb6-4501-bc53-a31f61798db0-crio-socket\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:21.519708 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.519682 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/610a5275-fbb6-4501-bc53-a31f61798db0-data-volume\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:21.519708 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:21.519707 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 00:44:21.519801 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:21.519747 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls podName:610a5275-fbb6-4501-bc53-a31f61798db0 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:22.019735832 +0000 UTC m=+64.820692922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bg6hz" (UID: "610a5275-fbb6-4501-bc53-a31f61798db0") : secret "insights-runtime-extractor-tls" not found Jul 30 00:44:21.528217 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.528197 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq8lx\" (UniqueName: \"kubernetes.io/projected/610a5275-fbb6-4501-bc53-a31f61798db0-kube-api-access-fq8lx\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:21.591861 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.591821 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:21.777425 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.777384 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:21.877624 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:21.877540 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:22.023387 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:22.023346 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:22.023583 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:22.023420 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:22.023583 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:22.023518 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 00:44:22.023583 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:22.023520 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 00:44:22.023583 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:22.023571 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm podName:610a5275-fbb6-4501-bc53-a31f61798db0 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:23.023557619 +0000 UTC m=+65.824514704 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bg6hz" (UID: "610a5275-fbb6-4501-bc53-a31f61798db0") : configmap "kube-rbac-proxy" not found Jul 30 00:44:22.023740 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:22.023597 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls podName:610a5275-fbb6-4501-bc53-a31f61798db0 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:23.023578515 +0000 UTC m=+65.824535598 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bg6hz" (UID: "610a5275-fbb6-4501-bc53-a31f61798db0") : secret "insights-runtime-extractor-tls" not found Jul 30 00:44:22.776348 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:22.776310 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:22.877467 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:22.877418 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:23.032541 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.032438 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:23.032541 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.032539 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:23.032727 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:23.032591 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 00:44:23.032727 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:23.032652 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls podName:610a5275-fbb6-4501-bc53-a31f61798db0 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:25.032638081 +0000 UTC m=+67.833595168 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bg6hz" (UID: "610a5275-fbb6-4501-bc53-a31f61798db0") : secret "insights-runtime-extractor-tls" not found Jul 30 00:44:23.032727 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:23.032657 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 00:44:23.032727 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:23.032700 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm podName:610a5275-fbb6-4501-bc53-a31f61798db0 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:25.032687622 +0000 UTC m=+67.833644711 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bg6hz" (UID: "610a5275-fbb6-4501-bc53-a31f61798db0") : configmap "kube-rbac-proxy" not found Jul 30 00:44:23.334779 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.334750 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs\") pod \"network-metrics-daemon-4r4tk\" (UID: \"4d304196-189e-44d8-ab5e-27ddbb8e18c2\") " pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:44:23.337119 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.337102 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 00:44:23.345849 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:23.345831 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 30 00:44:23.345937 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:23.345919 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs podName:4d304196-189e-44d8-ab5e-27ddbb8e18c2 nodeName:}" failed. No retries permitted until 2026-07-30 00:45:27.345887473 +0000 UTC m=+130.146844557 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs") pod "network-metrics-daemon-4r4tk" (UID: "4d304196-189e-44d8-ab5e-27ddbb8e18c2") : secret "metrics-daemon-secret" not found Jul 30 00:44:23.400110 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.400072 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:23.435356 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.435326 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:44:23.435476 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.435419 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjgm2\" (UniqueName: \"kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2\") pod \"network-check-target-z66tw\" (UID: \"0eb4b506-e95c-452e-a2f1-a0b683231704\") " pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:44:23.438036 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.438017 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 30 00:44:23.438135 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.438017 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 00:44:23.447623 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.447599 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 30 00:44:23.447887 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.447869 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/962582df-c2b5-4b1f-990a-18b7a1c3ccfd-original-pull-secret\") pod \"global-pull-secret-syncer-dj2qr\" (UID: \"962582df-c2b5-4b1f-990a-18b7a1c3ccfd\") " pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:44:23.458670 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.458647 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjgm2\" (UniqueName: \"kubernetes.io/projected/0eb4b506-e95c-452e-a2f1-a0b683231704-kube-api-access-vjgm2\") pod \"network-check-target-z66tw\" (UID: \"0eb4b506-e95c-452e-a2f1-a0b683231704\") " pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:44:23.584770 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.584732 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r24dv\"" Jul 30 00:44:23.588531 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.588492 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj2qr" Jul 30 00:44:23.592496 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.592483 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:44:23.718215 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.718187 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dj2qr"] Jul 30 00:44:23.722626 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:44:23.722603 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod962582df_c2b5_4b1f_990a_18b7a1c3ccfd.slice/crio-e1b77411e213f535cecbbfca1f1fbe127c7ef5373a6ca6a8c5d376f76aec4b4c WatchSource:0}: Error finding container e1b77411e213f535cecbbfca1f1fbe127c7ef5373a6ca6a8c5d376f76aec4b4c: Status 404 returned error can't find the container with id e1b77411e213f535cecbbfca1f1fbe127c7ef5373a6ca6a8c5d376f76aec4b4c Jul 30 00:44:23.735096 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.735069 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z66tw"] Jul 30 00:44:23.737833 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:44:23.737811 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eb4b506_e95c_452e_a2f1_a0b683231704.slice/crio-2edb4597fb815f3646487c28d55f5b25bda8439b02394a2abb3978a8d454ec9a WatchSource:0}: Error finding container 2edb4597fb815f3646487c28d55f5b25bda8439b02394a2abb3978a8d454ec9a: Status 404 returned error can't find the container with id 2edb4597fb815f3646487c28d55f5b25bda8439b02394a2abb3978a8d454ec9a Jul 30 00:44:23.893434 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.893338 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z66tw" event={"ID":"0eb4b506-e95c-452e-a2f1-a0b683231704","Type":"ContainerStarted","Data":"2edb4597fb815f3646487c28d55f5b25bda8439b02394a2abb3978a8d454ec9a"} Jul 30 00:44:23.894329 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:23.894309 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dj2qr" event={"ID":"962582df-c2b5-4b1f-990a-18b7a1c3ccfd","Type":"ContainerStarted","Data":"e1b77411e213f535cecbbfca1f1fbe127c7ef5373a6ca6a8c5d376f76aec4b4c"} Jul 30 00:44:24.104881 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:24.104853 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7xg42_61e4710e-5c0b-475d-80d4-db9f1cc78e27/dns-node-resolver/0.log" Jul 30 00:44:24.257568 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:24.257476 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:24.443206 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:24.443165 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:44:24.443415 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:24.443240 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:44:24.443415 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:24.443346 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 00:44:24.443415 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:24.443371 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56b77fd5b4-2zm7w: secret "image-registry-tls" not found Jul 30 00:44:24.443415 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:24.443392 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 00:44:24.443649 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:24.443432 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls podName:4af9a9e9-f2a5-4df9-aea5-7439becee5e1 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:56.443412434 +0000 UTC m=+99.244369525 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls") pod "image-registry-56b77fd5b4-2zm7w" (UID: "4af9a9e9-f2a5-4df9-aea5-7439becee5e1") : secret "image-registry-tls" not found Jul 30 00:44:24.443649 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:24.443450 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls podName:5c59f8b2-e65b-4f30-93bb-16a06e3d5f45 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:56.443442562 +0000 UTC m=+99.244399646 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls") pod "dns-default-8cxk4" (UID: "5c59f8b2-e65b-4f30-93bb-16a06e3d5f45") : secret "dns-default-metrics-tls" not found Jul 30 00:44:25.050230 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:25.050188 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:25.050701 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:25.050280 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:25.050701 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:25.050413 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 00:44:25.050701 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:25.050479 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls podName:610a5275-fbb6-4501-bc53-a31f61798db0 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:29.050459443 +0000 UTC m=+71.851416531 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bg6hz" (UID: "610a5275-fbb6-4501-bc53-a31f61798db0") : secret "insights-runtime-extractor-tls" not found Jul 30 00:44:25.051018 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:25.050863 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 00:44:25.051018 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:25.050943 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm podName:610a5275-fbb6-4501-bc53-a31f61798db0 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:29.050925037 +0000 UTC m=+71.851882138 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bg6hz" (UID: "610a5275-fbb6-4501-bc53-a31f61798db0") : configmap "kube-rbac-proxy" not found Jul 30 00:44:25.104520 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:25.104485 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xqfqf_aef50669-7ed3-432c-ad80-62254f6a5ac7/node-ca/0.log" Jul 30 00:44:25.704677 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:25.704603 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4t9fb_7d1f5f34-14d3-4458-9c22-f6b33bcea80e/serve-healthcheck-canary/0.log" Jul 30 00:44:26.782296 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:26.782263 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:27.289794 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:27.289756 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:27.473557 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:27.473516 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:27.573925 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:27.573858 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:27.747363 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:27.747323 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:27.848111 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:27.847959 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:27.983374 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:27.983340 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:28.909144 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:28.909106 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dj2qr" event={"ID":"962582df-c2b5-4b1f-990a-18b7a1c3ccfd","Type":"ContainerStarted","Data":"5af81abdc17020198cd91efbc74ae0c0e50fac724fbaf1b7b98aad485096fb68"} Jul 30 00:44:28.910457 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:28.910434 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z66tw" event={"ID":"0eb4b506-e95c-452e-a2f1-a0b683231704","Type":"ContainerStarted","Data":"873ea7a234824a1e1aaa2f3fb222e52d33f485c0167502e02dba04a9950a77d5"} Jul 30 00:44:28.910565 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:28.910555 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:44:28.924287 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:28.924230 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dj2qr" podStartSLOduration=66.158003888 podStartE2EDuration="1m10.924211868s" totalImagesPullingTime="4.76620798s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:28.923867315 +0000 UTC m=+71.724824424" watchObservedRunningTime="2026-07-30 00:44:28.924211868 +0000 UTC m=+71.725168974" Jul 30 00:44:28.937311 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:28.937255 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z66tw" podStartSLOduration=66.202910095 podStartE2EDuration="1m10.937236883s" totalImagesPullingTime="4.734326788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:28.936994158 +0000 UTC m=+71.737951267" watchObservedRunningTime="2026-07-30 00:44:28.937236883 +0000 UTC m=+71.738193989" Jul 30 00:44:28.939421 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:28.939397 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:28.995143 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:28.995113 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:29.087108 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:29.087075 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:29.087234 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:29.087152 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:29.087283 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:29.087239 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 00:44:29.087323 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:29.087293 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm podName:610a5275-fbb6-4501-bc53-a31f61798db0 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:37.087278748 +0000 UTC m=+79.888235832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bg6hz" (UID: "610a5275-fbb6-4501-bc53-a31f61798db0") : configmap "kube-rbac-proxy" not found Jul 30 00:44:29.087381 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:29.087238 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 00:44:29.087429 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:29.087404 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls podName:610a5275-fbb6-4501-bc53-a31f61798db0 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:37.087381836 +0000 UTC m=+79.888338921 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bg6hz" (UID: "610a5275-fbb6-4501-bc53-a31f61798db0") : secret "insights-runtime-extractor-tls" not found Jul 30 00:44:29.179853 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:29.179771 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:29.187790 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:29.187758 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:29.187957 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:29.187847 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:29.187957 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:29.187948 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle podName:49165c64-0239-4c5d-b252-218818d09280 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:45.187930457 +0000 UTC m=+87.988887541 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle") pod "router-default-67c4d7bd47-tszsf" (UID: "49165c64-0239-4c5d-b252-218818d09280") : configmap references non-existent config key: service-ca.crt Jul 30 00:44:29.188082 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:29.187991 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 00:44:29.188082 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:29.188050 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs podName:49165c64-0239-4c5d-b252-218818d09280 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:45.18803343 +0000 UTC m=+87.988990516 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs") pod "router-default-67c4d7bd47-tszsf" (UID: "49165c64-0239-4c5d-b252-218818d09280") : secret "router-metrics-certs-default" not found Jul 30 00:44:29.580811 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:29.580778 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:29.794720 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:29.794682 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:29.980962 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:29.980857 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:30.198884 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:30.198850 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:31.124729 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:31.124696 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:31.225493 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:31.225456 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:31.582738 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:31.582702 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:31.681090 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:31.681051 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:31.780712 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:31.780678 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:31.882002 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:31.881920 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:32.184195 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:32.184105 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:32.240348 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:32.240303 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:32.783280 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:32.783243 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:33.376943 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:33.376914 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:33.577275 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:33.577240 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:33.778919 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:33.778824 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:33.826361 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:33.826324 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:33.994314 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:33.994284 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:34.977701 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:34.977666 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:35.504756 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:35.504708 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:35.608953 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:35.608915 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:35.779149 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:35.779058 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:35.978951 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:35.978920 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:36.776614 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:36.776575 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:36.977187 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:36.977150 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:37.151324 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:37.151291 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:37.151685 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:37.151357 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:37.151685 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:37.151443 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 00:44:37.151685 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:37.151505 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls podName:610a5275-fbb6-4501-bc53-a31f61798db0 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:53.151488577 +0000 UTC m=+95.952445663 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bg6hz" (UID: "610a5275-fbb6-4501-bc53-a31f61798db0") : secret "insights-runtime-extractor-tls" not found Jul 30 00:44:37.151938 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:37.151923 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/610a5275-fbb6-4501-bc53-a31f61798db0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:37.381014 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:37.380973 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:37.780304 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:37.780273 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:37.982295 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:37.982258 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:39.675349 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:39.675205 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:39.971741 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:39.971654 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:44.123593 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.123553 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64"] Jul 30 00:44:44.128469 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.128445 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" Jul 30 00:44:44.131013 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.130990 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 30 00:44:44.131133 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.131041 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 30 00:44:44.132018 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.131999 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 30 00:44:44.132152 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.132132 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 30 00:44:44.142157 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.142137 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64"] Jul 30 00:44:44.209103 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.209046 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/cea14287-e030-4777-bf8d-f69bf0b22181-klusterlet-config\") pod \"klusterlet-addon-workmgr-85958dd9cc-bps64\" (UID: \"cea14287-e030-4777-bf8d-f69bf0b22181\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" Jul 30 00:44:44.209297 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.209180 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnjr8\" (UniqueName: \"kubernetes.io/projected/cea14287-e030-4777-bf8d-f69bf0b22181-kube-api-access-jnjr8\") pod \"klusterlet-addon-workmgr-85958dd9cc-bps64\" (UID: \"cea14287-e030-4777-bf8d-f69bf0b22181\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" Jul 30 00:44:44.209297 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.209217 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cea14287-e030-4777-bf8d-f69bf0b22181-tmp\") pod \"klusterlet-addon-workmgr-85958dd9cc-bps64\" (UID: \"cea14287-e030-4777-bf8d-f69bf0b22181\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" Jul 30 00:44:44.240541 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.240504 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:44.310117 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.310077 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cea14287-e030-4777-bf8d-f69bf0b22181-tmp\") pod \"klusterlet-addon-workmgr-85958dd9cc-bps64\" (UID: \"cea14287-e030-4777-bf8d-f69bf0b22181\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" Jul 30 00:44:44.310297 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.310146 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/cea14287-e030-4777-bf8d-f69bf0b22181-klusterlet-config\") pod \"klusterlet-addon-workmgr-85958dd9cc-bps64\" (UID: \"cea14287-e030-4777-bf8d-f69bf0b22181\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" Jul 30 00:44:44.310345 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.310321 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnjr8\" (UniqueName: \"kubernetes.io/projected/cea14287-e030-4777-bf8d-f69bf0b22181-kube-api-access-jnjr8\") pod \"klusterlet-addon-workmgr-85958dd9cc-bps64\" (UID: \"cea14287-e030-4777-bf8d-f69bf0b22181\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" Jul 30 00:44:44.310552 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.310532 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cea14287-e030-4777-bf8d-f69bf0b22181-tmp\") pod \"klusterlet-addon-workmgr-85958dd9cc-bps64\" (UID: \"cea14287-e030-4777-bf8d-f69bf0b22181\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" Jul 30 00:44:44.312620 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.312601 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/cea14287-e030-4777-bf8d-f69bf0b22181-klusterlet-config\") pod \"klusterlet-addon-workmgr-85958dd9cc-bps64\" (UID: \"cea14287-e030-4777-bf8d-f69bf0b22181\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" Jul 30 00:44:44.325518 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.325488 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnjr8\" (UniqueName: \"kubernetes.io/projected/cea14287-e030-4777-bf8d-f69bf0b22181-kube-api-access-jnjr8\") pod \"klusterlet-addon-workmgr-85958dd9cc-bps64\" (UID: \"cea14287-e030-4777-bf8d-f69bf0b22181\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" Jul 30 00:44:44.356793 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.356754 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:44.437463 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.437381 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" Jul 30 00:44:44.439518 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.439484 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:44.537470 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.537435 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:44.563322 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.563290 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64"] Jul 30 00:44:44.566654 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:44:44.566627 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcea14287_e030_4777_bf8d_f69bf0b22181.slice/crio-0e47339de465b765a0c7b04c71aefc04b6a78b4b43480107e882dc8360441f11 WatchSource:0}: Error finding container 0e47339de465b765a0c7b04c71aefc04b6a78b4b43480107e882dc8360441f11: Status 404 returned error can't find the container with id 0e47339de465b765a0c7b04c71aefc04b6a78b4b43480107e882dc8360441f11 Jul 30 00:44:44.637077 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.637040 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:44.736775 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.736664 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:44.948982 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:44.948950 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" event={"ID":"cea14287-e030-4777-bf8d-f69bf0b22181","Type":"ContainerStarted","Data":"0e47339de465b765a0c7b04c71aefc04b6a78b4b43480107e882dc8360441f11"} Jul 30 00:44:45.219875 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:45.219840 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:45.220355 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:45.219944 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:45.220500 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:45.220482 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49165c64-0239-4c5d-b252-218818d09280-service-ca-bundle\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:45.222305 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:45.222282 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49165c64-0239-4c5d-b252-218818d09280-metrics-certs\") pod \"router-default-67c4d7bd47-tszsf\" (UID: \"49165c64-0239-4c5d-b252-218818d09280\") " pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:45.411635 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:45.411600 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5k5jm\"" Jul 30 00:44:45.419413 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:45.419386 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:45.572445 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:45.572408 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-67c4d7bd47-tszsf"] Jul 30 00:44:45.576831 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:44:45.576799 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49165c64_0239_4c5d_b252_218818d09280.slice/crio-c8f59658cd69067cfd78e4a7714ddbb811baa9192ef53f868a2a715cb8f70563 WatchSource:0}: Error finding container c8f59658cd69067cfd78e4a7714ddbb811baa9192ef53f868a2a715cb8f70563: Status 404 returned error can't find the container with id c8f59658cd69067cfd78e4a7714ddbb811baa9192ef53f868a2a715cb8f70563 Jul 30 00:44:45.954440 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:45.954407 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-67c4d7bd47-tszsf" event={"ID":"49165c64-0239-4c5d-b252-218818d09280","Type":"ContainerStarted","Data":"fb96bbcf2a7a3f7acaedbf50d3e23bdada5777afb2ecf2123acb6e677637de99"} Jul 30 00:44:45.954637 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:45.954463 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-67c4d7bd47-tszsf" event={"ID":"49165c64-0239-4c5d-b252-218818d09280","Type":"ContainerStarted","Data":"c8f59658cd69067cfd78e4a7714ddbb811baa9192ef53f868a2a715cb8f70563"} Jul 30 00:44:45.972592 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:45.972534 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-67c4d7bd47-tszsf" podStartSLOduration=32.972512919 podStartE2EDuration="32.972512919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:45.971727981 +0000 UTC m=+88.772685087" watchObservedRunningTime="2026-07-30 00:44:45.972512919 +0000 UTC m=+88.773470020" Jul 30 00:44:46.420160 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:46.420125 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:46.422940 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:46.422917 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:46.957152 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:46.957117 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:46.958455 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:46.958431 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-67c4d7bd47-tszsf" Jul 30 00:44:48.592256 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:48.592214 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:48.753490 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:48.753400 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:48.936790 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:48.936751 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:48.963054 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:48.963021 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" event={"ID":"cea14287-e030-4777-bf8d-f69bf0b22181","Type":"ContainerStarted","Data":"d468f8f125a967446e292c4196d3410831f3f7034ccd27befe9736fbaf1a393f"} Jul 30 00:44:48.978659 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:48.978605 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" podStartSLOduration=1.164319803 podStartE2EDuration="4.978589956s" totalImagesPullingTime="3.814270153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:48.977030021 +0000 UTC m=+91.777987127" watchObservedRunningTime="2026-07-30 00:44:48.978589956 +0000 UTC m=+91.779547062" Jul 30 00:44:48.992658 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:48.992628 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:49.371681 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:49.371644 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:49.965275 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:49.965248 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" Jul 30 00:44:49.967094 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:49.967072 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85958dd9cc-bps64" Jul 30 00:44:52.772332 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:52.772294 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:52.840246 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:52.840210 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:52.892549 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:52.892515 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:53.188072 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:53.188015 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:53.190316 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:53.190285 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/610a5275-fbb6-4501-bc53-a31f61798db0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bg6hz\" (UID: \"610a5275-fbb6-4501-bc53-a31f61798db0\") " pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:53.427887 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:53.427854 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bg6hz" Jul 30 00:44:53.552729 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:53.552695 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bg6hz"] Jul 30 00:44:53.555891 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:44:53.555862 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod610a5275_fbb6_4501_bc53_a31f61798db0.slice/crio-7c6c762bbe61f211b88191c1df54a38b5d6eb869681ccd8a1dea2180cd9dee1d WatchSource:0}: Error finding container 7c6c762bbe61f211b88191c1df54a38b5d6eb869681ccd8a1dea2180cd9dee1d: Status 404 returned error can't find the container with id 7c6c762bbe61f211b88191c1df54a38b5d6eb869681ccd8a1dea2180cd9dee1d Jul 30 00:44:53.976091 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:53.976059 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bg6hz" event={"ID":"610a5275-fbb6-4501-bc53-a31f61798db0","Type":"ContainerStarted","Data":"7c6c762bbe61f211b88191c1df54a38b5d6eb869681ccd8a1dea2180cd9dee1d"} Jul 30 00:44:55.258921 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:55.258865 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:56.513612 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:56.513575 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:44:56.514068 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:56.513650 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:44:56.516434 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:56.516408 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c59f8b2-e65b-4f30-93bb-16a06e3d5f45-metrics-tls\") pod \"dns-default-8cxk4\" (UID: \"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45\") " pod="openshift-dns/dns-default-8cxk4" Jul 30 00:44:56.516566 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:56.516462 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls\") pod \"image-registry-56b77fd5b4-2zm7w\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:44:56.781673 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:56.781590 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zt6vm\"" Jul 30 00:44:56.789814 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:56.789717 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:44:56.795290 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:56.795257 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r2q5g\"" Jul 30 00:44:56.803092 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:56.803053 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8cxk4" Jul 30 00:44:57.899687 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:57.899642 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:58.133529 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.133491 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:58.146438 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.146410 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9"] Jul 30 00:44:58.149945 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.149870 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.152652 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.152583 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 30 00:44:58.152652 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.152633 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 30 00:44:58.152834 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.152584 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-bhv6b\"" Jul 30 00:44:58.153201 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.153178 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 30 00:44:58.153331 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.153218 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 30 00:44:58.153398 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.153326 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 30 00:44:58.153398 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.153387 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 30 00:44:58.162503 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.162477 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9"] Jul 30 00:44:58.178606 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.178567 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mt8qv"] Jul 30 00:44:58.181967 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.181938 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.184201 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.184177 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 30 00:44:58.184201 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.184198 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sjmm2\"" Jul 30 00:44:58.184465 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.184241 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 30 00:44:58.184465 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.184299 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 30 00:44:58.212474 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.212437 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:58.271239 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.271196 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:58.330128 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.330092 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jzh7\" (UniqueName: \"kubernetes.io/projected/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-kube-api-access-2jzh7\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.330128 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.330130 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-root\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.330361 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.330149 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-textfile\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.330361 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.330232 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-wtmp\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.330361 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.330268 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz68d\" (UniqueName: \"kubernetes.io/projected/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-kube-api-access-fz68d\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.330361 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.330296 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-sys\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.330523 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.330350 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.330523 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.330443 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-metrics-client-ca\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.330523 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.330464 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.330523 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.330480 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-tls\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.330649 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.330520 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.330649 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.330594 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.330649 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.330635 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.330755 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.330659 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.330755 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.330687 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.401343 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.401244 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:58.431433 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431358 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.431607 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431451 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.431607 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431485 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.431607 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431508 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jzh7\" (UniqueName: \"kubernetes.io/projected/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-kube-api-access-2jzh7\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.431607 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431530 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-root\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.431607 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431548 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-textfile\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.431607 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431580 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-wtmp\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.431607 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431603 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz68d\" (UniqueName: \"kubernetes.io/projected/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-kube-api-access-fz68d\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.431998 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431617 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-root\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.431998 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431632 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-sys\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.431998 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431660 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.431998 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431685 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-metrics-client-ca\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.431998 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431701 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.431998 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431718 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-tls\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.431998 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431744 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.431998 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431766 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.431998 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.431829 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-sys\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.432406 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.432182 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.432406 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.432195 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-textfile\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.432406 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.432222 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.432406 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:58.432249 2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 30 00:44:58.432406 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.432254 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.432406 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:58.432317 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-tls podName:2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:58.932296435 +0000 UTC m=+101.733253533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-tls") pod "node-exporter-mt8qv" (UID: "2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8") : secret "node-exporter-tls" not found Jul 30 00:44:58.432406 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.432320 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-wtmp\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.432406 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.432373 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.432747 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.432490 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-metrics-client-ca\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.434504 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.434477 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.434615 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.434579 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.434615 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.434596 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.440383 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.440354 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jzh7\" (UniqueName: \"kubernetes.io/projected/2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de-kube-api-access-2jzh7\") pod \"kube-state-metrics-56d9f944b4-qnlm9\" (UID: \"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.441489 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.441466 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz68d\" (UniqueName: \"kubernetes.io/projected/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-kube-api-access-fz68d\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.460607 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.460574 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" Jul 30 00:44:58.513446 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.511769 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:58.598455 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.598372 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8cxk4"] Jul 30 00:44:58.601088 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.601046 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:58.602738 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:44:58.602710 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c59f8b2_e65b_4f30_93bb_16a06e3d5f45.slice/crio-306d38a16a5559afad6620906186f386759d87b372d44c46cc372bd919270ff0 WatchSource:0}: Error finding container 306d38a16a5559afad6620906186f386759d87b372d44c46cc372bd919270ff0: Status 404 returned error can't find the container with id 306d38a16a5559afad6620906186f386759d87b372d44c46cc372bd919270ff0 Jul 30 00:44:58.621126 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.621043 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56b77fd5b4-2zm7w"] Jul 30 00:44:58.626712 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:44:58.626680 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4af9a9e9_f2a5_4df9_aea5_7439becee5e1.slice/crio-95f85c514aa861542499ecfaf57ce9b4a0f1cee5f5494cf871dd620f22852d2f WatchSource:0}: Error finding container 95f85c514aa861542499ecfaf57ce9b4a0f1cee5f5494cf871dd620f22852d2f: Status 404 returned error can't find the container with id 95f85c514aa861542499ecfaf57ce9b4a0f1cee5f5494cf871dd620f22852d2f Jul 30 00:44:58.649659 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.649631 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9"] Jul 30 00:44:58.653222 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.653127 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:58.654107 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:44:58.654079 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c3e8ce9_db86_42a0_a06e_cd2ecc75d6de.slice/crio-f98ebcfaf468fdaa3805ae784260055a49943d2bd045b34780885b2522004b55 WatchSource:0}: Error finding container f98ebcfaf468fdaa3805ae784260055a49943d2bd045b34780885b2522004b55: Status 404 returned error can't find the container with id f98ebcfaf468fdaa3805ae784260055a49943d2bd045b34780885b2522004b55 Jul 30 00:44:58.936245 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.936144 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-tls\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.938532 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.938506 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8-node-exporter-tls\") pod \"node-exporter-mt8qv\" (UID: \"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8\") " pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:58.995113 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.995074 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" event={"ID":"4af9a9e9-f2a5-4df9-aea5-7439becee5e1","Type":"ContainerStarted","Data":"1cf4713e701f4b598be2aac98f4fb48ce90c693d890326e127b05c7e48e8f90c"} Jul 30 00:44:58.995293 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.995132 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" event={"ID":"4af9a9e9-f2a5-4df9-aea5-7439becee5e1","Type":"ContainerStarted","Data":"95f85c514aa861542499ecfaf57ce9b4a0f1cee5f5494cf871dd620f22852d2f"} Jul 30 00:44:58.995293 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.995174 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:44:58.996351 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.996327 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8cxk4" event={"ID":"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45","Type":"ContainerStarted","Data":"306d38a16a5559afad6620906186f386759d87b372d44c46cc372bd919270ff0"} Jul 30 00:44:58.997939 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.997911 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bg6hz" event={"ID":"610a5275-fbb6-4501-bc53-a31f61798db0","Type":"ContainerStarted","Data":"1b6fcdddb9bc7b8111f80238f9a646da5af5b075f594f08bf1e0573b51ad4803"} Jul 30 00:44:58.999079 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:58.999056 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" event={"ID":"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de","Type":"ContainerStarted","Data":"f98ebcfaf468fdaa3805ae784260055a49943d2bd045b34780885b2522004b55"} Jul 30 00:44:59.013462 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.013397 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" podStartSLOduration=100.01337972 podStartE2EDuration="1m40.01337972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:43:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:59.012345799 +0000 UTC m=+101.813302905" watchObservedRunningTime="2026-07-30 00:44:59.01337972 +0000 UTC m=+101.814336826" Jul 30 00:44:59.093276 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.093009 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mt8qv" Jul 30 00:44:59.105674 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:44:59.105644 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d9cec99_d40d_43a2_a4f5_3359a4d0b3a8.slice/crio-d3f2dfd865fa7eba6814cc2e988559e11d3519775f276e7589c910ce737c5be1 WatchSource:0}: Error finding container d3f2dfd865fa7eba6814cc2e988559e11d3519775f276e7589c910ce737c5be1: Status 404 returned error can't find the container with id d3f2dfd865fa7eba6814cc2e988559e11d3519775f276e7589c910ce737c5be1 Jul 30 00:44:59.123334 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.123167 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:59.198245 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.196256 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 00:44:59.202267 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.201243 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.202267 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.201344 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:59.204974 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.204947 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 30 00:44:59.205291 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.205261 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 30 00:44:59.205466 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.205449 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4vsfq\"" Jul 30 00:44:59.205684 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.205666 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 30 00:44:59.205890 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.205876 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 30 00:44:59.206136 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.206121 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 30 00:44:59.206340 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.206300 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 30 00:44:59.206495 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.206469 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 30 00:44:59.206648 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.206633 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 30 00:44:59.206886 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.206859 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 30 00:44:59.218731 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.217616 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 00:44:59.340439 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.340403 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2d84b5-73ab-47af-a82a-3f5a4a370234-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.340608 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.340447 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-web-config\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.340608 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.340475 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf2r8\" (UniqueName: \"kubernetes.io/projected/1c2d84b5-73ab-47af-a82a-3f5a4a370234-kube-api-access-sf2r8\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.340608 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.340502 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c2d84b5-73ab-47af-a82a-3f5a4a370234-config-out\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.340608 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.340526 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c2d84b5-73ab-47af-a82a-3f5a4a370234-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.340608 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.340557 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.340608 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.340605 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1c2d84b5-73ab-47af-a82a-3f5a4a370234-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.340940 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.340633 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.340940 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.340661 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c2d84b5-73ab-47af-a82a-3f5a4a370234-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.340940 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.340688 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.340940 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.340729 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.340940 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.340756 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.340940 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.340787 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-config-volume\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.445913 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.445855 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-config-volume\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.446070 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.445969 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2d84b5-73ab-47af-a82a-3f5a4a370234-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.446070 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.445998 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-web-config\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.446070 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.446023 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf2r8\" (UniqueName: \"kubernetes.io/projected/1c2d84b5-73ab-47af-a82a-3f5a4a370234-kube-api-access-sf2r8\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.446070 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.446048 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c2d84b5-73ab-47af-a82a-3f5a4a370234-config-out\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.446291 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.446074 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c2d84b5-73ab-47af-a82a-3f5a4a370234-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.446291 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.446108 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.446291 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.446245 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1c2d84b5-73ab-47af-a82a-3f5a4a370234-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.446291 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.446289 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.446487 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.446318 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c2d84b5-73ab-47af-a82a-3f5a4a370234-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.446487 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.446346 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.446487 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.446388 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.446487 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.446416 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.450265 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.449876 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.451997 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.451396 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c2d84b5-73ab-47af-a82a-3f5a4a370234-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.451997 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.451672 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1c2d84b5-73ab-47af-a82a-3f5a4a370234-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.455163 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.454653 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.455163 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:59.454751 2564 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 30 00:44:59.455163 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:59.454825 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-main-tls podName:1c2d84b5-73ab-47af-a82a-3f5a4a370234 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:59.954802549 +0000 UTC m=+102.755759636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "1c2d84b5-73ab-47af-a82a-3f5a4a370234") : secret "alertmanager-main-tls" not found Jul 30 00:44:59.455163 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:44:59.455130 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1c2d84b5-73ab-47af-a82a-3f5a4a370234-alertmanager-trusted-ca-bundle podName:1c2d84b5-73ab-47af-a82a-3f5a4a370234 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:59.955113624 +0000 UTC m=+102.756070708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/1c2d84b5-73ab-47af-a82a-3f5a4a370234-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "1c2d84b5-73ab-47af-a82a-3f5a4a370234") : configmap references non-existent config key: ca-bundle.crt Jul 30 00:44:59.457644 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.457556 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf2r8\" (UniqueName: \"kubernetes.io/projected/1c2d84b5-73ab-47af-a82a-3f5a4a370234-kube-api-access-sf2r8\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.460997 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.460819 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c2d84b5-73ab-47af-a82a-3f5a4a370234-config-out\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.460997 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.460883 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c2d84b5-73ab-47af-a82a-3f5a4a370234-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.460997 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.460939 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.462955 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.462912 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.463196 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.463153 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-web-config\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.466328 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.466304 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-config-volume\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:44:59.556391 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.556330 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:59.607181 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.607056 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:44:59.917125 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:44:59.916423 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z66tw" Jul 30 00:45:00.006773 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:00.006689 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mt8qv" event={"ID":"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8","Type":"ContainerStarted","Data":"d3f2dfd865fa7eba6814cc2e988559e11d3519775f276e7589c910ce737c5be1"} Jul 30 00:45:00.051617 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:00.051573 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2d84b5-73ab-47af-a82a-3f5a4a370234-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:45:00.051819 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:00.051695 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:45:00.052658 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:00.052600 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2d84b5-73ab-47af-a82a-3f5a4a370234-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:45:00.055909 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:00.055866 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:45:00.093656 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:00.093616 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:00.121372 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:00.121285 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:45:01.087161 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.087123 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:01.137128 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.134296 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-78b977d57-mrbbw"] Jul 30 00:45:01.160047 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.160008 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78b977d57-mrbbw"] Jul 30 00:45:01.160247 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.160152 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.162985 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.162958 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 30 00:45:01.162985 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.162970 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 30 00:45:01.163241 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.163203 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 30 00:45:01.163373 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.163262 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 30 00:45:01.163468 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.163453 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 30 00:45:01.163639 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.163625 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-s7egnkq9qd7m\"" Jul 30 00:45:01.164077 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.163980 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-6sqpr\"" Jul 30 00:45:01.187661 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.187619 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:01.262841 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.262770 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.262841 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.262826 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-thanos-querier-tls\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.263099 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.262910 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4dwd\" (UniqueName: \"kubernetes.io/projected/2a889c73-df8d-4baf-bb64-8bc313d57684-kube-api-access-w4dwd\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.263099 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.262981 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.263099 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.263017 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-grpc-tls\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.263099 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.263050 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a889c73-df8d-4baf-bb64-8bc313d57684-metrics-client-ca\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.263275 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.263170 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.263275 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.263220 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.363680 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.363595 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.363680 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.363643 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.363680 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.363667 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.363992 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.363693 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-thanos-querier-tls\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.363992 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.363711 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4dwd\" (UniqueName: \"kubernetes.io/projected/2a889c73-df8d-4baf-bb64-8bc313d57684-kube-api-access-w4dwd\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.363992 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.363740 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.363992 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.363766 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-grpc-tls\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.363992 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.363797 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a889c73-df8d-4baf-bb64-8bc313d57684-metrics-client-ca\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.364577 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.364548 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a889c73-df8d-4baf-bb64-8bc313d57684-metrics-client-ca\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.366491 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.366454 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.366634 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.366616 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-grpc-tls\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.366708 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.366638 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.366763 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.366711 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-thanos-querier-tls\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.366763 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.366708 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.366763 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.366746 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2a889c73-df8d-4baf-bb64-8bc313d57684-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.373113 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.373089 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4dwd\" (UniqueName: \"kubernetes.io/projected/2a889c73-df8d-4baf-bb64-8bc313d57684-kube-api-access-w4dwd\") pod \"thanos-querier-78b977d57-mrbbw\" (UID: \"2a889c73-df8d-4baf-bb64-8bc313d57684\") " pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.421676 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.421640 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:01.471870 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.471835 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:01.595923 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:01.595875 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:02.441435 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.441398 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5d7dd9c977-8bdmd"] Jul 30 00:45:02.585095 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.585051 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d7dd9c977-8bdmd"] Jul 30 00:45:02.585095 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.585086 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.588093 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.588069 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 30 00:45:02.588875 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.588844 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9jc96\"" Jul 30 00:45:02.588875 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.588870 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 30 00:45:02.589066 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.588855 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5tbms5qpfkt6p\"" Jul 30 00:45:02.589066 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.588929 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 30 00:45:02.589066 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.588849 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 30 00:45:02.777203 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.777103 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/969cb85c-a22f-4477-a864-75194e9af1da-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.777203 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.777170 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/969cb85c-a22f-4477-a864-75194e9af1da-client-ca-bundle\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.777470 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.777282 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfzkv\" (UniqueName: \"kubernetes.io/projected/969cb85c-a22f-4477-a864-75194e9af1da-kube-api-access-bfzkv\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.777470 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.777340 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/969cb85c-a22f-4477-a864-75194e9af1da-secret-metrics-server-tls\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.777470 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.777412 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/969cb85c-a22f-4477-a864-75194e9af1da-metrics-server-audit-profiles\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.777617 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.777474 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/969cb85c-a22f-4477-a864-75194e9af1da-secret-metrics-server-client-certs\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.777617 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.777534 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/969cb85c-a22f-4477-a864-75194e9af1da-audit-log\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.878174 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.878137 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfzkv\" (UniqueName: \"kubernetes.io/projected/969cb85c-a22f-4477-a864-75194e9af1da-kube-api-access-bfzkv\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.878341 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.878192 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/969cb85c-a22f-4477-a864-75194e9af1da-secret-metrics-server-tls\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.878341 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.878222 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/969cb85c-a22f-4477-a864-75194e9af1da-metrics-server-audit-profiles\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.878341 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.878257 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/969cb85c-a22f-4477-a864-75194e9af1da-secret-metrics-server-client-certs\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.878510 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.878353 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/969cb85c-a22f-4477-a864-75194e9af1da-audit-log\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.878510 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.878407 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/969cb85c-a22f-4477-a864-75194e9af1da-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.878510 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.878447 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/969cb85c-a22f-4477-a864-75194e9af1da-client-ca-bundle\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.879039 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.879012 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/969cb85c-a22f-4477-a864-75194e9af1da-audit-log\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.879732 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.879647 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/969cb85c-a22f-4477-a864-75194e9af1da-metrics-server-audit-profiles\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.879854 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.879730 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/969cb85c-a22f-4477-a864-75194e9af1da-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.881253 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.881206 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/969cb85c-a22f-4477-a864-75194e9af1da-secret-metrics-server-tls\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.881392 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.881371 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/969cb85c-a22f-4477-a864-75194e9af1da-client-ca-bundle\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.881965 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.881926 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/969cb85c-a22f-4477-a864-75194e9af1da-secret-metrics-server-client-certs\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.889735 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.889704 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfzkv\" (UniqueName: \"kubernetes.io/projected/969cb85c-a22f-4477-a864-75194e9af1da-kube-api-access-bfzkv\") pod \"metrics-server-5d7dd9c977-8bdmd\" (UID: \"969cb85c-a22f-4477-a864-75194e9af1da\") " pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.897566 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.897536 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:02.926734 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.926702 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c"] Jul 30 00:45:02.961353 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.961317 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c"] Jul 30 00:45:02.961496 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.961329 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c" Jul 30 00:45:02.963761 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.963737 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 30 00:45:02.963879 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:02.963737 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-w24bq\"" Jul 30 00:45:03.019919 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:03.019870 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:03.080149 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:03.080117 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce77146e-a447-4a19-a1c6-b9fffbac4144-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-jbr7c\" (UID: \"ce77146e-a447-4a19-a1c6-b9fffbac4144\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c" Jul 30 00:45:03.080300 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:03.080181 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5xdq\" (UniqueName: \"kubernetes.io/projected/ce77146e-a447-4a19-a1c6-b9fffbac4144-kube-api-access-f5xdq\") pod \"monitoring-plugin-68b7fc9574-jbr7c\" (UID: \"ce77146e-a447-4a19-a1c6-b9fffbac4144\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c" Jul 30 00:45:03.181100 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:03.181063 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5xdq\" (UniqueName: \"kubernetes.io/projected/ce77146e-a447-4a19-a1c6-b9fffbac4144-kube-api-access-f5xdq\") pod \"monitoring-plugin-68b7fc9574-jbr7c\" (UID: \"ce77146e-a447-4a19-a1c6-b9fffbac4144\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c" Jul 30 00:45:03.181290 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:03.181195 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce77146e-a447-4a19-a1c6-b9fffbac4144-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-jbr7c\" (UID: \"ce77146e-a447-4a19-a1c6-b9fffbac4144\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c" Jul 30 00:45:03.183803 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:03.183768 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce77146e-a447-4a19-a1c6-b9fffbac4144-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-jbr7c\" (UID: \"ce77146e-a447-4a19-a1c6-b9fffbac4144\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c" Jul 30 00:45:03.188744 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:03.188690 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5xdq\" (UniqueName: \"kubernetes.io/projected/ce77146e-a447-4a19-a1c6-b9fffbac4144-kube-api-access-f5xdq\") pod \"monitoring-plugin-68b7fc9574-jbr7c\" (UID: \"ce77146e-a447-4a19-a1c6-b9fffbac4144\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c" Jul 30 00:45:03.198236 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:03.198203 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:03.271137 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:03.271108 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c" Jul 30 00:45:03.853991 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:03.852006 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 00:45:03.927476 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:03.927089 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:03.972053 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:03.971690 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d7dd9c977-8bdmd"] Jul 30 00:45:04.005108 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:04.004508 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78b977d57-mrbbw"] Jul 30 00:45:04.010975 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:45:04.010943 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a889c73_df8d_4baf_bb64_8bc313d57684.slice/crio-7777b8b9983872920eaf039fb97fbfbc22eed260c94c4acb8378bcef626fc686 WatchSource:0}: Error finding container 7777b8b9983872920eaf039fb97fbfbc22eed260c94c4acb8378bcef626fc686: Status 404 returned error can't find the container with id 7777b8b9983872920eaf039fb97fbfbc22eed260c94c4acb8378bcef626fc686 Jul 30 00:45:04.029580 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:04.028137 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" event={"ID":"969cb85c-a22f-4477-a864-75194e9af1da","Type":"ContainerStarted","Data":"9c97f41d697910b0ceb77e8ac5e58cacff212ede2360f89172a522e84d6e1b46"} Jul 30 00:45:04.053181 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:04.053136 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" event={"ID":"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de","Type":"ContainerStarted","Data":"fbbc18ac4cd56be5ef91b6ee0fb18aefc1009653b98e1007ac1a0c10320e71b7"} Jul 30 00:45:04.057777 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:04.057204 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mt8qv" event={"ID":"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8","Type":"ContainerStarted","Data":"111d5d0ee952f00181fbd18865c1e090296e225f5578ac6a74adea79bf6562b0"} Jul 30 00:45:04.065626 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:04.063088 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" event={"ID":"2a889c73-df8d-4baf-bb64-8bc313d57684","Type":"ContainerStarted","Data":"7777b8b9983872920eaf039fb97fbfbc22eed260c94c4acb8378bcef626fc686"} Jul 30 00:45:04.068235 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:04.067986 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c2d84b5-73ab-47af-a82a-3f5a4a370234","Type":"ContainerStarted","Data":"3bb0be8a512b8e0ae181a5259cfb179de48dda8669d8ce2ade5b4b774042b26f"} Jul 30 00:45:04.203468 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:04.203434 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c"] Jul 30 00:45:04.206961 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:45:04.206933 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce77146e_a447_4a19_a1c6_b9fffbac4144.slice/crio-cc87f9d94fba55945b1d31d1cd3cbf8c3e74d9bf358f10145d15cad3a472811e WatchSource:0}: Error finding container cc87f9d94fba55945b1d31d1cd3cbf8c3e74d9bf358f10145d15cad3a472811e: Status 404 returned error can't find the container with id cc87f9d94fba55945b1d31d1cd3cbf8c3e74d9bf358f10145d15cad3a472811e Jul 30 00:45:04.380504 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:04.380464 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:05.079098 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:05.079056 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bg6hz" event={"ID":"610a5275-fbb6-4501-bc53-a31f61798db0","Type":"ContainerStarted","Data":"07920bc65d8b6401ba48f1ac8d55dbe8708239253fab2ecc2b37a85069488ca6"} Jul 30 00:45:05.083379 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:05.082378 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" event={"ID":"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de","Type":"ContainerStarted","Data":"fe332c4d95d6e56c5fefb936e5e08c12ac3befb421f1c4467652b212484084d2"} Jul 30 00:45:05.083379 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:05.082437 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" event={"ID":"2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de","Type":"ContainerStarted","Data":"71700c1fc7b610bafdb8bf24579b6e66b2256621bc66804319188888eaca457a"} Jul 30 00:45:05.086356 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:05.085442 2564 generic.go:356] "Generic (PLEG): container finished" podID="2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8" containerID="111d5d0ee952f00181fbd18865c1e090296e225f5578ac6a74adea79bf6562b0" exitCode=0 Jul 30 00:45:05.086356 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:05.085529 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mt8qv" event={"ID":"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8","Type":"ContainerDied","Data":"111d5d0ee952f00181fbd18865c1e090296e225f5578ac6a74adea79bf6562b0"} Jul 30 00:45:05.090921 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:05.090855 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8cxk4" event={"ID":"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45","Type":"ContainerStarted","Data":"647f649615910c7dd6af5333c0efdb41d262d58765d6e353703a81f6db2a0ee7"} Jul 30 00:45:05.091197 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:05.091033 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8cxk4" event={"ID":"5c59f8b2-e65b-4f30-93bb-16a06e3d5f45","Type":"ContainerStarted","Data":"599e9b5ab3fe61a88a0ef67868ee851e9db99c9472fc52d45d3f84d3fd0dc896"} Jul 30 00:45:05.091298 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:05.091247 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8cxk4" Jul 30 00:45:05.092269 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:05.092248 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c" event={"ID":"ce77146e-a447-4a19-a1c6-b9fffbac4144","Type":"ContainerStarted","Data":"cc87f9d94fba55945b1d31d1cd3cbf8c3e74d9bf358f10145d15cad3a472811e"} Jul 30 00:45:05.103505 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:05.102736 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-qnlm9" podStartSLOduration=2.091568066 podStartE2EDuration="7.10271442s" totalImagesPullingTime="5.011146354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:05.10123045 +0000 UTC m=+107.902187567" watchObservedRunningTime="2026-07-30 00:45:05.10271442 +0000 UTC m=+107.903671530" Jul 30 00:45:05.891684 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:05.891441 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8cxk4" podStartSLOduration=68.713704824 podStartE2EDuration="1m13.891417315s" totalImagesPullingTime="5.177712491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:43:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:05.134678118 +0000 UTC m=+107.935635227" watchObservedRunningTime="2026-07-30 00:45:05.891417315 +0000 UTC m=+108.692374422" Jul 30 00:45:05.897765 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:05.893400 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56b77fd5b4-2zm7w"] Jul 30 00:45:05.972571 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:05.972540 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:06.075047 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:06.075003 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:06.099006 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:06.098921 2564 generic.go:356] "Generic (PLEG): container finished" podID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerID="6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140" exitCode=0 Jul 30 00:45:06.099006 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:06.098958 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c2d84b5-73ab-47af-a82a-3f5a4a370234","Type":"ContainerDied","Data":"6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140"} Jul 30 00:45:06.102977 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:06.102948 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mt8qv" event={"ID":"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8","Type":"ContainerStarted","Data":"54f5152301664d0b6b0da1d79e94f781263f73a0ff8ab682004b9e186d4f9041"} Jul 30 00:45:09.117565 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.116555 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mt8qv" event={"ID":"2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8","Type":"ContainerStarted","Data":"d32255ab9a9c97e42c01d292d02d149e2dc2d572892c9c636b8047a8746d8c14"} Jul 30 00:45:09.118603 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.118568 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-nc5kn"] Jul 30 00:45:09.122155 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.122132 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-nc5kn" Jul 30 00:45:09.124327 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.124292 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 30 00:45:09.124327 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.124306 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 30 00:45:09.124522 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.124292 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gvwcr\"" Jul 30 00:45:09.134003 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.133961 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-nc5kn"] Jul 30 00:45:09.141516 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.140970 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w9qv\" (UniqueName: \"kubernetes.io/projected/7f5a55fe-584b-402f-8850-758a3367d187-kube-api-access-2w9qv\") pod \"downloads-7c97dc4555-nc5kn\" (UID: \"7f5a55fe-584b-402f-8850-758a3367d187\") " pod="openshift-console/downloads-7c97dc4555-nc5kn" Jul 30 00:45:09.142336 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.142282 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mt8qv" podStartSLOduration=6.454485482 podStartE2EDuration="11.142261384s" totalImagesPullingTime="4.555726938s" totalInitContainerRuntime="132.048964ms" isStatefulPod=true podCreationTimestamp="2026-07-30 00:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:09.140395684 +0000 UTC m=+111.941352800" watchObservedRunningTime="2026-07-30 00:45:09.142261384 +0000 UTC m=+111.943218491" Jul 30 00:45:09.183434 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.183293 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:09.242196 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.242168 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w9qv\" (UniqueName: \"kubernetes.io/projected/7f5a55fe-584b-402f-8850-758a3367d187-kube-api-access-2w9qv\") pod \"downloads-7c97dc4555-nc5kn\" (UID: \"7f5a55fe-584b-402f-8850-758a3367d187\") " pod="openshift-console/downloads-7c97dc4555-nc5kn" Jul 30 00:45:09.249888 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.249865 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w9qv\" (UniqueName: \"kubernetes.io/projected/7f5a55fe-584b-402f-8850-758a3367d187-kube-api-access-2w9qv\") pod \"downloads-7c97dc4555-nc5kn\" (UID: \"7f5a55fe-584b-402f-8850-758a3367d187\") " pod="openshift-console/downloads-7c97dc4555-nc5kn" Jul 30 00:45:09.284083 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.283962 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:09.434728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.434676 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-nc5kn" Jul 30 00:45:09.588331 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.588306 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-nc5kn"] Jul 30 00:45:09.591948 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:45:09.591918 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f5a55fe_584b_402f_8850_758a3367d187.slice/crio-e290306502bb5a720c746eedb12ce5e7733c68fb9c6318efa7a51e70f41d9830 WatchSource:0}: Error finding container e290306502bb5a720c746eedb12ce5e7733c68fb9c6318efa7a51e70f41d9830: Status 404 returned error can't find the container with id e290306502bb5a720c746eedb12ce5e7733c68fb9c6318efa7a51e70f41d9830 Jul 30 00:45:09.639413 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.639314 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:09.801123 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.801090 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:09.858629 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:09.858600 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:10.124064 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.124026 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" event={"ID":"2a889c73-df8d-4baf-bb64-8bc313d57684","Type":"ContainerStarted","Data":"975ef1965774c36b9afc52164424e113a73934eddf176d5c62b10f6474f6eed0"} Jul 30 00:45:10.124577 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.124101 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" event={"ID":"2a889c73-df8d-4baf-bb64-8bc313d57684","Type":"ContainerStarted","Data":"cdf14fb03c213ebb24f1d09aa3860f00b75dec70cb976486c79efd40bb20d5fe"} Jul 30 00:45:10.124577 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.124138 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" event={"ID":"2a889c73-df8d-4baf-bb64-8bc313d57684","Type":"ContainerStarted","Data":"ab81ed7cf77cd72103e26c695938aec95be8924e85cfeca6b33635a406763910"} Jul 30 00:45:10.127746 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.127705 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c2d84b5-73ab-47af-a82a-3f5a4a370234","Type":"ContainerStarted","Data":"d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595"} Jul 30 00:45:10.127885 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.127775 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c2d84b5-73ab-47af-a82a-3f5a4a370234","Type":"ContainerStarted","Data":"2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655"} Jul 30 00:45:10.127885 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.127817 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c2d84b5-73ab-47af-a82a-3f5a4a370234","Type":"ContainerStarted","Data":"030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d"} Jul 30 00:45:10.127885 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.127854 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c2d84b5-73ab-47af-a82a-3f5a4a370234","Type":"ContainerStarted","Data":"a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc"} Jul 30 00:45:10.128020 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.127889 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c2d84b5-73ab-47af-a82a-3f5a4a370234","Type":"ContainerStarted","Data":"0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c"} Jul 30 00:45:10.129480 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.129453 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c" event={"ID":"ce77146e-a447-4a19-a1c6-b9fffbac4144","Type":"ContainerStarted","Data":"298ffc728e1a00767c3971474edc88f2275611403d308a38ca89b86f3d93377f"} Jul 30 00:45:10.129632 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.129609 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c" Jul 30 00:45:10.131077 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.131054 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-nc5kn" event={"ID":"7f5a55fe-584b-402f-8850-758a3367d187","Type":"ContainerStarted","Data":"e290306502bb5a720c746eedb12ce5e7733c68fb9c6318efa7a51e70f41d9830"} Jul 30 00:45:10.133003 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.132952 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" event={"ID":"969cb85c-a22f-4477-a864-75194e9af1da","Type":"ContainerStarted","Data":"2c715ac5e6a036d806ff3899d55f3f238338e878285c89ef4dec607977122c4f"} Jul 30 00:45:10.135631 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.135613 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c" Jul 30 00:45:10.136289 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.135835 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bg6hz" event={"ID":"610a5275-fbb6-4501-bc53-a31f61798db0","Type":"ContainerStarted","Data":"8b2b9ac3371576ea0d13e413c4539515514d7a53215674faa470a7dd53acc4cf"} Jul 30 00:45:10.145581 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.145533 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jbr7c" podStartSLOduration=3.5791493819999998 podStartE2EDuration="8.145517033s" totalImagesPullingTime="4.566367651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:10.144208609 +0000 UTC m=+112.945165716" watchObservedRunningTime="2026-07-30 00:45:10.145517033 +0000 UTC m=+112.946474131" Jul 30 00:45:10.160301 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.160245 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" podStartSLOduration=3.3643878369999998 podStartE2EDuration="8.160227932s" totalImagesPullingTime="4.795840095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:10.159607604 +0000 UTC m=+112.960564711" watchObservedRunningTime="2026-07-30 00:45:10.160227932 +0000 UTC m=+112.961185038" Jul 30 00:45:10.200086 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.198967 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bg6hz" podStartSLOduration=33.826686839 podStartE2EDuration="49.198946023s" totalImagesPullingTime="15.372259184s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:44:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:10.198489837 +0000 UTC m=+112.999446945" watchObservedRunningTime="2026-07-30 00:45:10.198946023 +0000 UTC m=+112.999903130" Jul 30 00:45:10.230830 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.230797 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:10.283203 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:10.283164 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:11.015202 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:11.015165 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:11.143371 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:11.143327 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:11.144557 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:11.144528 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" event={"ID":"2a889c73-df8d-4baf-bb64-8bc313d57684","Type":"ContainerStarted","Data":"3182a021e045233d034c34332bd8da947644401c2a214347bdc35cad4d0bbad8"} Jul 30 00:45:11.144676 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:11.144637 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" event={"ID":"2a889c73-df8d-4baf-bb64-8bc313d57684","Type":"ContainerStarted","Data":"135d4469b3a24dfbf0192d6b4708f8f3b273594c14106946a4211a85852a818d"} Jul 30 00:45:11.144756 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:11.144680 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" event={"ID":"2a889c73-df8d-4baf-bb64-8bc313d57684","Type":"ContainerStarted","Data":"97cc363083575933e5b409607fd6e1aaa9acf7024566e2e886d789fb431d4c40"} Jul 30 00:45:11.144756 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:11.144721 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:11.148485 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:11.148437 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c2d84b5-73ab-47af-a82a-3f5a4a370234","Type":"ContainerStarted","Data":"46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4"} Jul 30 00:45:11.166627 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:11.166567 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" podStartSLOduration=4.50210988 podStartE2EDuration="10.166543411s" totalImagesPullingTime="5.664433531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:45:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:11.165441433 +0000 UTC m=+113.966398531" watchObservedRunningTime="2026-07-30 00:45:11.166543411 +0000 UTC m=+113.967500509" Jul 30 00:45:11.192274 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:11.192188 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.704776806 podStartE2EDuration="12.192164688s" totalImagesPullingTime="5.468674034s" totalInitContainerRuntime="18.713848ms" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:11.190635038 +0000 UTC m=+113.991592147" watchObservedRunningTime="2026-07-30 00:45:11.192164688 +0000 UTC m=+113.993121796" Jul 30 00:45:11.244959 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:11.244922 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:12.426650 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:12.426591 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:12.479132 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:12.479084 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:14.609155 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:14.609088 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:14.660332 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:14.660270 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:15.900039 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:15.900012 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:45:16.109040 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:16.109012 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8cxk4" Jul 30 00:45:16.110725 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:16.110694 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:16.211584 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:16.211499 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:17.158850 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:17.158822 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-78b977d57-mrbbw" Jul 30 00:45:17.164484 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:17.164413 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:17.264705 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:17.264670 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:17.365641 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:17.365605 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:18.448331 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.448280 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7544d5ccf9-b9pmw"] Jul 30 00:45:18.453870 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.453840 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.457015 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.456989 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wjdhw\"" Jul 30 00:45:18.457146 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.457032 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 30 00:45:18.457146 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.457053 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 30 00:45:18.457146 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.456994 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 30 00:45:18.457146 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.457047 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 30 00:45:18.457497 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.456989 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 30 00:45:18.463691 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.463000 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7544d5ccf9-b9pmw"] Jul 30 00:45:18.527947 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.527908 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cd710a5-3058-4286-a8a1-16d798aab259-console-serving-cert\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.528132 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.527987 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cd710a5-3058-4286-a8a1-16d798aab259-console-oauth-config\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.528132 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.528010 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-oauth-serving-cert\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.528132 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.528043 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-service-ca\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.528132 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.528072 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-console-config\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.528132 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.528109 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmgnk\" (UniqueName: \"kubernetes.io/projected/6cd710a5-3058-4286-a8a1-16d798aab259-kube-api-access-xmgnk\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.629208 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.629164 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cd710a5-3058-4286-a8a1-16d798aab259-console-oauth-config\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.629208 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.629207 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-oauth-serving-cert\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.629536 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.629239 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-service-ca\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.629536 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.629266 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-console-config\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.629536 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.629309 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmgnk\" (UniqueName: \"kubernetes.io/projected/6cd710a5-3058-4286-a8a1-16d798aab259-kube-api-access-xmgnk\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.629536 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.629380 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cd710a5-3058-4286-a8a1-16d798aab259-console-serving-cert\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.630122 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.630070 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-oauth-serving-cert\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.630229 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.630149 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-console-config\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.630364 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.630341 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-service-ca\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.632270 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.632239 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cd710a5-3058-4286-a8a1-16d798aab259-console-oauth-config\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.632451 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.632418 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cd710a5-3058-4286-a8a1-16d798aab259-console-serving-cert\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.640633 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.640599 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmgnk\" (UniqueName: \"kubernetes.io/projected/6cd710a5-3058-4286-a8a1-16d798aab259-kube-api-access-xmgnk\") pod \"console-7544d5ccf9-b9pmw\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.766887 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.766811 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:18.932873 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:18.932844 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7544d5ccf9-b9pmw"] Jul 30 00:45:18.935584 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:45:18.935551 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd710a5_3058_4286_a8a1_16d798aab259.slice/crio-1dae19e7fb395aa7719620c68bec032b15ad830acd2700821ff4ea7c297efc49 WatchSource:0}: Error finding container 1dae19e7fb395aa7719620c68bec032b15ad830acd2700821ff4ea7c297efc49: Status 404 returned error can't find the container with id 1dae19e7fb395aa7719620c68bec032b15ad830acd2700821ff4ea7c297efc49 Jul 30 00:45:19.186105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:19.186066 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7544d5ccf9-b9pmw" event={"ID":"6cd710a5-3058-4286-a8a1-16d798aab259","Type":"ContainerStarted","Data":"1dae19e7fb395aa7719620c68bec032b15ad830acd2700821ff4ea7c297efc49"} Jul 30 00:45:22.898457 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:22.898415 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:22.899081 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:22.898526 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:23.677844 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:23.677784 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:23.778283 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:23.778243 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:24.031597 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:24.031515 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:24.676426 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:24.676390 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:24.726319 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:24.726280 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:25.218566 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:25.218528 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:25.270779 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:25.270741 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:27.413590 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:27.413549 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs\") pod \"network-metrics-daemon-4r4tk\" (UID: \"4d304196-189e-44d8-ab5e-27ddbb8e18c2\") " pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:45:27.416278 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:27.416245 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d304196-189e-44d8-ab5e-27ddbb8e18c2-metrics-certs\") pod \"network-metrics-daemon-4r4tk\" (UID: \"4d304196-189e-44d8-ab5e-27ddbb8e18c2\") " pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:45:27.431507 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:27.431470 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:27.478003 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:27.477969 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vdmkk\"" Jul 30 00:45:27.485732 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:27.485695 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r4tk" Jul 30 00:45:27.490136 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:27.490098 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:29.228960 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:29.228925 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7544d5ccf9-b9pmw" event={"ID":"6cd710a5-3058-4286-a8a1-16d798aab259","Type":"ContainerStarted","Data":"a62993ab76dcc81daaa875c139f0a4134c4fc29c450efee84686429963c08768"} Jul 30 00:45:29.230634 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:29.230596 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-nc5kn" event={"ID":"7f5a55fe-584b-402f-8850-758a3367d187","Type":"ContainerStarted","Data":"fb31fa77a33b41d8ed1033658e1f52c517a2cfa1aa7a978489d2838864d531ce"} Jul 30 00:45:29.230805 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:29.230792 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-nc5kn" Jul 30 00:45:29.232341 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:29.232314 2564 patch_prober.go:28] interesting pod/downloads-7c97dc4555-nc5kn container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.17:8080/\": dial tcp 10.132.0.17:8080: connect: connection refused" start-of-body= Jul 30 00:45:29.232444 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:29.232364 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-nc5kn" podUID="7f5a55fe-584b-402f-8850-758a3367d187" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.17:8080/\": dial tcp 10.132.0.17:8080: connect: connection refused" Jul 30 00:45:29.237796 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:29.237775 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4r4tk"] Jul 30 00:45:29.241562 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:45:29.241495 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d304196_189e_44d8_ab5e_27ddbb8e18c2.slice/crio-f48ad239d995dcee23fe4748cea11c0779fb8b85652112252b98f8011a6a9d3a WatchSource:0}: Error finding container f48ad239d995dcee23fe4748cea11c0779fb8b85652112252b98f8011a6a9d3a: Status 404 returned error can't find the container with id f48ad239d995dcee23fe4748cea11c0779fb8b85652112252b98f8011a6a9d3a Jul 30 00:45:29.247140 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:29.247095 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7544d5ccf9-b9pmw" podStartSLOduration=1.072752706 podStartE2EDuration="11.247078099s" totalImagesPullingTime="10.174325393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:45:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:29.24610001 +0000 UTC m=+132.047057131" watchObservedRunningTime="2026-07-30 00:45:29.247078099 +0000 UTC m=+132.048035183" Jul 30 00:45:29.262293 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:29.262241 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-nc5kn" podStartSLOduration=0.720321674 podStartE2EDuration="20.262222758s" totalImagesPullingTime="19.541901084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:29.26157673 +0000 UTC m=+132.062533837" watchObservedRunningTime="2026-07-30 00:45:29.262222758 +0000 UTC m=+132.063179864" Jul 30 00:45:30.237798 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:30.237679 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4r4tk" event={"ID":"4d304196-189e-44d8-ab5e-27ddbb8e18c2","Type":"ContainerStarted","Data":"f48ad239d995dcee23fe4748cea11c0779fb8b85652112252b98f8011a6a9d3a"} Jul 30 00:45:30.242978 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:30.242914 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-nc5kn" Jul 30 00:45:30.433822 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:30.433727 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:30.483755 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:30.483716 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:30.921381 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:30.921334 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" podUID="4af9a9e9-f2a5-4df9-aea5-7439becee5e1" containerName="registry" containerID="cri-o://1cf4713e701f4b598be2aac98f4fb48ce90c693d890326e127b05c7e48e8f90c" gracePeriod=30 Jul 30 00:45:31.245019 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.244937 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4r4tk" event={"ID":"4d304196-189e-44d8-ab5e-27ddbb8e18c2","Type":"ContainerStarted","Data":"9f2a109ace185e3b815829ffd6e0e9bee64490a9bc84e4c4be664b239a963c95"} Jul 30 00:45:31.246839 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.246807 2564 generic.go:356] "Generic (PLEG): container finished" podID="4af9a9e9-f2a5-4df9-aea5-7439becee5e1" containerID="1cf4713e701f4b598be2aac98f4fb48ce90c693d890326e127b05c7e48e8f90c" exitCode=0 Jul 30 00:45:31.246978 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.246924 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" event={"ID":"4af9a9e9-f2a5-4df9-aea5-7439becee5e1","Type":"ContainerDied","Data":"1cf4713e701f4b598be2aac98f4fb48ce90c693d890326e127b05c7e48e8f90c"} Jul 30 00:45:31.247046 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.246975 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" event={"ID":"4af9a9e9-f2a5-4df9-aea5-7439becee5e1","Type":"ContainerDied","Data":"95f85c514aa861542499ecfaf57ce9b4a0f1cee5f5494cf871dd620f22852d2f"} Jul 30 00:45:31.247046 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.246997 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95f85c514aa861542499ecfaf57ce9b4a0f1cee5f5494cf871dd620f22852d2f" Jul 30 00:45:31.269972 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.269934 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:45:31.349922 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.349867 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls\") pod \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " Jul 30 00:45:31.350119 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.349951 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-bound-sa-token\") pod \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " Jul 30 00:45:31.350119 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.350014 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-installation-pull-secrets\") pod \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " Jul 30 00:45:31.350119 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.350061 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-ca-trust-extracted\") pod \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " Jul 30 00:45:31.350119 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.350092 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-certificates\") pod \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " Jul 30 00:45:31.350119 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.350120 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9pg5\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-kube-api-access-f9pg5\") pod \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " Jul 30 00:45:31.350379 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.350149 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-trusted-ca\") pod \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " Jul 30 00:45:31.350379 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.350209 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-image-registry-private-configuration\") pod \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\" (UID: \"4af9a9e9-f2a5-4df9-aea5-7439becee5e1\") " Jul 30 00:45:31.350672 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.350633 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4af9a9e9-f2a5-4df9-aea5-7439becee5e1" (UID: "4af9a9e9-f2a5-4df9-aea5-7439becee5e1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:45:31.351642 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.351605 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4af9a9e9-f2a5-4df9-aea5-7439becee5e1" (UID: "4af9a9e9-f2a5-4df9-aea5-7439becee5e1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:45:31.354022 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.353976 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-kube-api-access-f9pg5" (OuterVolumeSpecName: "kube-api-access-f9pg5") pod "4af9a9e9-f2a5-4df9-aea5-7439becee5e1" (UID: "4af9a9e9-f2a5-4df9-aea5-7439becee5e1"). InnerVolumeSpecName "kube-api-access-f9pg5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:45:31.354378 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.354350 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4af9a9e9-f2a5-4df9-aea5-7439becee5e1" (UID: "4af9a9e9-f2a5-4df9-aea5-7439becee5e1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:45:31.354487 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.354424 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4af9a9e9-f2a5-4df9-aea5-7439becee5e1" (UID: "4af9a9e9-f2a5-4df9-aea5-7439becee5e1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:45:31.355177 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.355145 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4af9a9e9-f2a5-4df9-aea5-7439becee5e1" (UID: "4af9a9e9-f2a5-4df9-aea5-7439becee5e1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:45:31.356048 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.355910 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4af9a9e9-f2a5-4df9-aea5-7439becee5e1" (UID: "4af9a9e9-f2a5-4df9-aea5-7439becee5e1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:45:31.362441 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.362399 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4af9a9e9-f2a5-4df9-aea5-7439becee5e1" (UID: "4af9a9e9-f2a5-4df9-aea5-7439becee5e1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:45:31.451597 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.451556 2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-installation-pull-secrets\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:45:31.451597 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.451596 2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-ca-trust-extracted\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:45:31.451819 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.451608 2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-certificates\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:45:31.451819 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.451620 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9pg5\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-kube-api-access-f9pg5\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:45:31.451819 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.451632 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-trusted-ca\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:45:31.451819 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.451646 2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-image-registry-private-configuration\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:45:31.451819 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.451658 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-registry-tls\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:45:31.451819 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:31.451669 2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4af9a9e9-f2a5-4df9-aea5-7439becee5e1-bound-sa-token\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:45:32.253006 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:32.252965 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4r4tk" event={"ID":"4d304196-189e-44d8-ab5e-27ddbb8e18c2","Type":"ContainerStarted","Data":"80d2a191435bf327e13f574fe1b552f1c1027bff81463ef8aaef8c70ac9d0dae"} Jul 30 00:45:32.253491 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:32.253022 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56b77fd5b4-2zm7w" Jul 30 00:45:32.269598 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:32.269534 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4r4tk" podStartSLOduration=132.581364867 podStartE2EDuration="2m14.269513643s" totalImagesPullingTime="1.688148776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:32.268142783 +0000 UTC m=+135.069099889" watchObservedRunningTime="2026-07-30 00:45:32.269513643 +0000 UTC m=+135.070470748" Jul 30 00:45:32.283126 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:32.283092 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56b77fd5b4-2zm7w"] Jul 30 00:45:32.287141 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:32.287102 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-56b77fd5b4-2zm7w"] Jul 30 00:45:32.359819 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:32.359775 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:32.462422 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:32.462382 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:33.670118 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:33.670084 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4af9a9e9-f2a5-4df9-aea5-7439becee5e1" path="/var/lib/kubelet/pods/4af9a9e9-f2a5-4df9-aea5-7439becee5e1/volumes" Jul 30 00:45:36.149660 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:36.149618 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:36.205180 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:36.205140 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:37.059721 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:37.059685 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7544d5ccf9-b9pmw"] Jul 30 00:45:37.157402 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:37.157364 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:37.258388 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:37.258306 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:37.359623 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:37.359587 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:38.767225 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:38.767187 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:45:39.678142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:39.678110 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:39.778085 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:39.778046 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:42.903764 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:42.903738 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:42.907555 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:42.907532 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5d7dd9c977-8bdmd" Jul 30 00:45:54.670405 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:54.670365 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:54.730153 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:45:54.730116 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:02.082495 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.082435 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7544d5ccf9-b9pmw" podUID="6cd710a5-3058-4286-a8a1-16d798aab259" containerName="console" containerID="cri-o://a62993ab76dcc81daaa875c139f0a4134c4fc29c450efee84686429963c08768" gracePeriod=15 Jul 30 00:46:02.375212 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.375176 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:46:02.378769 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.378740 2564 generic.go:356] "Generic (PLEG): container finished" podID="6cd710a5-3058-4286-a8a1-16d798aab259" containerID="a62993ab76dcc81daaa875c139f0a4134c4fc29c450efee84686429963c08768" exitCode=0 Jul 30 00:46:02.378950 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.378777 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7544d5ccf9-b9pmw" Jul 30 00:46:02.378950 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.378824 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7544d5ccf9-b9pmw" event={"ID":"6cd710a5-3058-4286-a8a1-16d798aab259","Type":"ContainerDied","Data":"a62993ab76dcc81daaa875c139f0a4134c4fc29c450efee84686429963c08768"} Jul 30 00:46:02.378950 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.378881 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7544d5ccf9-b9pmw" event={"ID":"6cd710a5-3058-4286-a8a1-16d798aab259","Type":"ContainerDied","Data":"1dae19e7fb395aa7719620c68bec032b15ad830acd2700821ff4ea7c297efc49"} Jul 30 00:46:02.378950 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.378933 2564 scope.go:117] "RemoveContainer" containerID="a62993ab76dcc81daaa875c139f0a4134c4fc29c450efee84686429963c08768" Jul 30 00:46:02.389587 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.389558 2564 scope.go:117] "RemoveContainer" containerID="a62993ab76dcc81daaa875c139f0a4134c4fc29c450efee84686429963c08768" Jul 30 00:46:02.389881 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:46:02.389847 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a62993ab76dcc81daaa875c139f0a4134c4fc29c450efee84686429963c08768\": container with ID starting with a62993ab76dcc81daaa875c139f0a4134c4fc29c450efee84686429963c08768 not found: ID does not exist" containerID="a62993ab76dcc81daaa875c139f0a4134c4fc29c450efee84686429963c08768" Jul 30 00:46:02.389943 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.389912 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a62993ab76dcc81daaa875c139f0a4134c4fc29c450efee84686429963c08768"} err="failed to get container status \"a62993ab76dcc81daaa875c139f0a4134c4fc29c450efee84686429963c08768\": rpc error: code = NotFound desc = could not find container \"a62993ab76dcc81daaa875c139f0a4134c4fc29c450efee84686429963c08768\": container with ID starting with a62993ab76dcc81daaa875c139f0a4134c4fc29c450efee84686429963c08768 not found: ID does not exist" Jul 30 00:46:02.411157 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.411125 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmgnk\" (UniqueName: \"kubernetes.io/projected/6cd710a5-3058-4286-a8a1-16d798aab259-kube-api-access-xmgnk\") pod \"6cd710a5-3058-4286-a8a1-16d798aab259\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " Jul 30 00:46:02.411345 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.411192 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-oauth-serving-cert\") pod \"6cd710a5-3058-4286-a8a1-16d798aab259\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " Jul 30 00:46:02.411345 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.411230 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cd710a5-3058-4286-a8a1-16d798aab259-console-oauth-config\") pod \"6cd710a5-3058-4286-a8a1-16d798aab259\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " Jul 30 00:46:02.411345 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.411258 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cd710a5-3058-4286-a8a1-16d798aab259-console-serving-cert\") pod \"6cd710a5-3058-4286-a8a1-16d798aab259\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " Jul 30 00:46:02.411345 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.411273 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-service-ca\") pod \"6cd710a5-3058-4286-a8a1-16d798aab259\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " Jul 30 00:46:02.411345 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.411288 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-console-config\") pod \"6cd710a5-3058-4286-a8a1-16d798aab259\" (UID: \"6cd710a5-3058-4286-a8a1-16d798aab259\") " Jul 30 00:46:02.411611 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.411584 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6cd710a5-3058-4286-a8a1-16d798aab259" (UID: "6cd710a5-3058-4286-a8a1-16d798aab259"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:46:02.411742 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.411718 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-service-ca" (OuterVolumeSpecName: "service-ca") pod "6cd710a5-3058-4286-a8a1-16d798aab259" (UID: "6cd710a5-3058-4286-a8a1-16d798aab259"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:46:02.411811 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.411745 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-console-config" (OuterVolumeSpecName: "console-config") pod "6cd710a5-3058-4286-a8a1-16d798aab259" (UID: "6cd710a5-3058-4286-a8a1-16d798aab259"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:46:02.413543 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.413514 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cd710a5-3058-4286-a8a1-16d798aab259-kube-api-access-xmgnk" (OuterVolumeSpecName: "kube-api-access-xmgnk") pod "6cd710a5-3058-4286-a8a1-16d798aab259" (UID: "6cd710a5-3058-4286-a8a1-16d798aab259"). InnerVolumeSpecName "kube-api-access-xmgnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:46:02.413650 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.413562 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cd710a5-3058-4286-a8a1-16d798aab259-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6cd710a5-3058-4286-a8a1-16d798aab259" (UID: "6cd710a5-3058-4286-a8a1-16d798aab259"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:46:02.413650 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.413575 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cd710a5-3058-4286-a8a1-16d798aab259-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6cd710a5-3058-4286-a8a1-16d798aab259" (UID: "6cd710a5-3058-4286-a8a1-16d798aab259"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:46:02.511978 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.511935 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmgnk\" (UniqueName: \"kubernetes.io/projected/6cd710a5-3058-4286-a8a1-16d798aab259-kube-api-access-xmgnk\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:02.511978 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.511967 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-oauth-serving-cert\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:02.511978 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.511978 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cd710a5-3058-4286-a8a1-16d798aab259-console-oauth-config\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:02.511978 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.511987 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cd710a5-3058-4286-a8a1-16d798aab259-console-serving-cert\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:02.512253 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.511996 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-service-ca\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:02.512253 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.512004 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cd710a5-3058-4286-a8a1-16d798aab259-console-config\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:02.699726 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.699688 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7544d5ccf9-b9pmw"] Jul 30 00:46:02.704962 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:02.704934 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7544d5ccf9-b9pmw"] Jul 30 00:46:03.668437 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:03.668405 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cd710a5-3058-4286-a8a1-16d798aab259" path="/var/lib/kubelet/pods/6cd710a5-3058-4286-a8a1-16d798aab259/volumes" Jul 30 00:46:17.341298 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:17.341211 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:17.648399 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:17.648307 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:18.388590 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:18.388561 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 00:46:18.389017 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:18.388848 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="alertmanager" containerID="cri-o://0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c" gracePeriod=120 Jul 30 00:46:18.389017 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:18.388888 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="kube-rbac-proxy-metric" containerID="cri-o://d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595" gracePeriod=120 Jul 30 00:46:18.389017 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:18.388955 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="kube-rbac-proxy" containerID="cri-o://2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655" gracePeriod=120 Jul 30 00:46:18.389017 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:18.388978 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="kube-rbac-proxy-web" containerID="cri-o://030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d" gracePeriod=120 Jul 30 00:46:18.389231 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:18.388989 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="config-reloader" containerID="cri-o://a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc" gracePeriod=120 Jul 30 00:46:18.389231 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:18.388964 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="prom-label-proxy" containerID="cri-o://46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4" gracePeriod=120 Jul 30 00:46:19.454036 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.454003 2564 generic.go:356] "Generic (PLEG): container finished" podID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerID="46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4" exitCode=0 Jul 30 00:46:19.454036 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.454029 2564 generic.go:356] "Generic (PLEG): container finished" podID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerID="2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655" exitCode=0 Jul 30 00:46:19.454036 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.454037 2564 generic.go:356] "Generic (PLEG): container finished" podID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerID="a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc" exitCode=0 Jul 30 00:46:19.454036 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.454043 2564 generic.go:356] "Generic (PLEG): container finished" podID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerID="0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c" exitCode=0 Jul 30 00:46:19.454494 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.454076 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c2d84b5-73ab-47af-a82a-3f5a4a370234","Type":"ContainerDied","Data":"46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4"} Jul 30 00:46:19.454494 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.454160 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c2d84b5-73ab-47af-a82a-3f5a4a370234","Type":"ContainerDied","Data":"2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655"} Jul 30 00:46:19.454494 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.454203 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c2d84b5-73ab-47af-a82a-3f5a4a370234","Type":"ContainerDied","Data":"a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc"} Jul 30 00:46:19.454494 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.454226 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c2d84b5-73ab-47af-a82a-3f5a4a370234","Type":"ContainerDied","Data":"0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c"} Jul 30 00:46:19.631632 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.631607 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:19.753207 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.753117 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-main-tls\") pod \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " Jul 30 00:46:19.753207 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.753168 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " Jul 30 00:46:19.753207 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.753191 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1c2d84b5-73ab-47af-a82a-3f5a4a370234-alertmanager-main-db\") pod \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " Jul 30 00:46:19.753488 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.753305 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c2d84b5-73ab-47af-a82a-3f5a4a370234-config-out\") pod \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " Jul 30 00:46:19.753488 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.753338 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c2d84b5-73ab-47af-a82a-3f5a4a370234-tls-assets\") pod \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " Jul 30 00:46:19.753488 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.753399 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy-web\") pod \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " Jul 30 00:46:19.753488 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.753434 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-cluster-tls-config\") pod \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " Jul 30 00:46:19.753488 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.753470 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sf2r8\" (UniqueName: \"kubernetes.io/projected/1c2d84b5-73ab-47af-a82a-3f5a4a370234-kube-api-access-sf2r8\") pod \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " Jul 30 00:46:19.753739 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.753498 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c2d84b5-73ab-47af-a82a-3f5a4a370234-metrics-client-ca\") pod \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " Jul 30 00:46:19.753739 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.753515 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c2d84b5-73ab-47af-a82a-3f5a4a370234-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1c2d84b5-73ab-47af-a82a-3f5a4a370234" (UID: "1c2d84b5-73ab-47af-a82a-3f5a4a370234"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:46:19.753739 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.753543 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-config-volume\") pod \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " Jul 30 00:46:19.753739 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.753596 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-web-config\") pod \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " Jul 30 00:46:19.753739 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.753622 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy\") pod \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " Jul 30 00:46:19.753739 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.753670 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2d84b5-73ab-47af-a82a-3f5a4a370234-alertmanager-trusted-ca-bundle\") pod \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\" (UID: \"1c2d84b5-73ab-47af-a82a-3f5a4a370234\") " Jul 30 00:46:19.754031 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.754013 2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1c2d84b5-73ab-47af-a82a-3f5a4a370234-alertmanager-main-db\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:19.755519 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.754640 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c2d84b5-73ab-47af-a82a-3f5a4a370234-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1c2d84b5-73ab-47af-a82a-3f5a4a370234" (UID: "1c2d84b5-73ab-47af-a82a-3f5a4a370234"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:46:19.756146 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.756120 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c2d84b5-73ab-47af-a82a-3f5a4a370234-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1c2d84b5-73ab-47af-a82a-3f5a4a370234" (UID: "1c2d84b5-73ab-47af-a82a-3f5a4a370234"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:46:19.756342 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.756320 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1c2d84b5-73ab-47af-a82a-3f5a4a370234" (UID: "1c2d84b5-73ab-47af-a82a-3f5a4a370234"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:46:19.756342 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.756261 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c2d84b5-73ab-47af-a82a-3f5a4a370234-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1c2d84b5-73ab-47af-a82a-3f5a4a370234" (UID: "1c2d84b5-73ab-47af-a82a-3f5a4a370234"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:46:19.756490 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.756463 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1c2d84b5-73ab-47af-a82a-3f5a4a370234" (UID: "1c2d84b5-73ab-47af-a82a-3f5a4a370234"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:46:19.756556 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.756488 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1c2d84b5-73ab-47af-a82a-3f5a4a370234" (UID: "1c2d84b5-73ab-47af-a82a-3f5a4a370234"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:46:19.756617 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.756589 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c2d84b5-73ab-47af-a82a-3f5a4a370234-config-out" (OuterVolumeSpecName: "config-out") pod "1c2d84b5-73ab-47af-a82a-3f5a4a370234" (UID: "1c2d84b5-73ab-47af-a82a-3f5a4a370234"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:46:19.756868 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.756842 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1c2d84b5-73ab-47af-a82a-3f5a4a370234" (UID: "1c2d84b5-73ab-47af-a82a-3f5a4a370234"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:46:19.757060 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.757034 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c2d84b5-73ab-47af-a82a-3f5a4a370234-kube-api-access-sf2r8" (OuterVolumeSpecName: "kube-api-access-sf2r8") pod "1c2d84b5-73ab-47af-a82a-3f5a4a370234" (UID: "1c2d84b5-73ab-47af-a82a-3f5a4a370234"). InnerVolumeSpecName "kube-api-access-sf2r8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:46:19.757563 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.757543 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-config-volume" (OuterVolumeSpecName: "config-volume") pod "1c2d84b5-73ab-47af-a82a-3f5a4a370234" (UID: "1c2d84b5-73ab-47af-a82a-3f5a4a370234"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:46:19.762143 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.762115 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1c2d84b5-73ab-47af-a82a-3f5a4a370234" (UID: "1c2d84b5-73ab-47af-a82a-3f5a4a370234"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:46:19.768254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.768229 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-web-config" (OuterVolumeSpecName: "web-config") pod "1c2d84b5-73ab-47af-a82a-3f5a4a370234" (UID: "1c2d84b5-73ab-47af-a82a-3f5a4a370234"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:46:19.854427 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.854387 2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-web-config\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:19.854427 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.854419 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:19.854427 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.854430 2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2d84b5-73ab-47af-a82a-3f5a4a370234-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:19.854427 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.854440 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-main-tls\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:19.854688 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.854451 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:19.854688 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.854461 2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c2d84b5-73ab-47af-a82a-3f5a4a370234-config-out\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:19.854688 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.854469 2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c2d84b5-73ab-47af-a82a-3f5a4a370234-tls-assets\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:19.854688 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.854477 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:19.854688 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.854485 2564 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-cluster-tls-config\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:19.854688 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.854494 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sf2r8\" (UniqueName: \"kubernetes.io/projected/1c2d84b5-73ab-47af-a82a-3f5a4a370234-kube-api-access-sf2r8\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:19.854688 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.854502 2564 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c2d84b5-73ab-47af-a82a-3f5a4a370234-metrics-client-ca\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:19.854688 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:19.854509 2564 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1c2d84b5-73ab-47af-a82a-3f5a4a370234-config-volume\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:46:20.461060 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.461024 2564 generic.go:356] "Generic (PLEG): container finished" podID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerID="d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595" exitCode=0 Jul 30 00:46:20.461060 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.461052 2564 generic.go:356] "Generic (PLEG): container finished" podID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerID="030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d" exitCode=0 Jul 30 00:46:20.461494 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.461096 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.461494 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.461100 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c2d84b5-73ab-47af-a82a-3f5a4a370234","Type":"ContainerDied","Data":"d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595"} Jul 30 00:46:20.461494 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.461178 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c2d84b5-73ab-47af-a82a-3f5a4a370234","Type":"ContainerDied","Data":"030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d"} Jul 30 00:46:20.461494 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.461207 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c2d84b5-73ab-47af-a82a-3f5a4a370234","Type":"ContainerDied","Data":"3bb0be8a512b8e0ae181a5259cfb179de48dda8669d8ce2ade5b4b774042b26f"} Jul 30 00:46:20.461494 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.461251 2564 scope.go:117] "RemoveContainer" containerID="46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4" Jul 30 00:46:20.468855 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.468754 2564 scope.go:117] "RemoveContainer" containerID="d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595" Jul 30 00:46:20.476253 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.476233 2564 scope.go:117] "RemoveContainer" containerID="2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655" Jul 30 00:46:20.483627 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.483536 2564 scope.go:117] "RemoveContainer" containerID="030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d" Jul 30 00:46:20.484584 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.484560 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 00:46:20.488472 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.488444 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 00:46:20.491358 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.491341 2564 scope.go:117] "RemoveContainer" containerID="a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc" Jul 30 00:46:20.498854 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.498831 2564 scope.go:117] "RemoveContainer" containerID="0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c" Jul 30 00:46:20.506865 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.506708 2564 scope.go:117] "RemoveContainer" containerID="6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140" Jul 30 00:46:20.514713 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.514689 2564 scope.go:117] "RemoveContainer" containerID="46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4" Jul 30 00:46:20.515213 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:46:20.515188 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4\": container with ID starting with 46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4 not found: ID does not exist" containerID="46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4" Jul 30 00:46:20.515307 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.515224 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4"} err="failed to get container status \"46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4\": rpc error: code = NotFound desc = could not find container \"46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4\": container with ID starting with 46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4 not found: ID does not exist" Jul 30 00:46:20.515307 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.515251 2564 scope.go:117] "RemoveContainer" containerID="d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595" Jul 30 00:46:20.515568 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:46:20.515541 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595\": container with ID starting with d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595 not found: ID does not exist" containerID="d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595" Jul 30 00:46:20.515616 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.515573 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595"} err="failed to get container status \"d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595\": rpc error: code = NotFound desc = could not find container \"d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595\": container with ID starting with d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595 not found: ID does not exist" Jul 30 00:46:20.515616 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.515588 2564 scope.go:117] "RemoveContainer" containerID="2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655" Jul 30 00:46:20.515869 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:46:20.515847 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655\": container with ID starting with 2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655 not found: ID does not exist" containerID="2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655" Jul 30 00:46:20.515937 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.515876 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655"} err="failed to get container status \"2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655\": rpc error: code = NotFound desc = could not find container \"2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655\": container with ID starting with 2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655 not found: ID does not exist" Jul 30 00:46:20.515937 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.515892 2564 scope.go:117] "RemoveContainer" containerID="030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d" Jul 30 00:46:20.516126 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516109 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 00:46:20.516186 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:46:20.516172 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d\": container with ID starting with 030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d not found: ID does not exist" containerID="030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d" Jul 30 00:46:20.516229 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516190 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d"} err="failed to get container status \"030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d\": rpc error: code = NotFound desc = could not find container \"030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d\": container with ID starting with 030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d not found: ID does not exist" Jul 30 00:46:20.516229 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516204 2564 scope.go:117] "RemoveContainer" containerID="a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc" Jul 30 00:46:20.516418 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516404 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cd710a5-3058-4286-a8a1-16d798aab259" containerName="console" Jul 30 00:46:20.516472 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:46:20.516410 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc\": container with ID starting with a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc not found: ID does not exist" containerID="a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc" Jul 30 00:46:20.516472 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516443 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc"} err="failed to get container status \"a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc\": rpc error: code = NotFound desc = could not find container \"a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc\": container with ID starting with a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc not found: ID does not exist" Jul 30 00:46:20.516472 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516465 2564 scope.go:117] "RemoveContainer" containerID="0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c" Jul 30 00:46:20.516595 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516419 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cd710a5-3058-4286-a8a1-16d798aab259" containerName="console" Jul 30 00:46:20.516595 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516504 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="kube-rbac-proxy" Jul 30 00:46:20.516595 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516510 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="kube-rbac-proxy" Jul 30 00:46:20.516595 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516518 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="alertmanager" Jul 30 00:46:20.516595 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516523 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="alertmanager" Jul 30 00:46:20.516595 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516531 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="kube-rbac-proxy-metric" Jul 30 00:46:20.516595 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516538 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="kube-rbac-proxy-metric" Jul 30 00:46:20.516595 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516550 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="init-config-reloader" Jul 30 00:46:20.516595 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516559 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="init-config-reloader" Jul 30 00:46:20.516595 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516581 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="kube-rbac-proxy-web" Jul 30 00:46:20.516595 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516590 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="kube-rbac-proxy-web" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516601 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="config-reloader" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516609 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="config-reloader" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516623 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="prom-label-proxy" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516631 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="prom-label-proxy" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516642 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4af9a9e9-f2a5-4df9-aea5-7439becee5e1" containerName="registry" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516650 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4af9a9e9-f2a5-4df9-aea5-7439becee5e1" containerName="registry" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:46:20.516705 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c\": container with ID starting with 0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c not found: ID does not exist" containerID="0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516721 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="config-reloader" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516721 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c"} err="failed to get container status \"0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c\": rpc error: code = NotFound desc = could not find container \"0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c\": container with ID starting with 0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c not found: ID does not exist" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516730 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="kube-rbac-proxy-metric" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516735 2564 scope.go:117] "RemoveContainer" containerID="6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516737 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="prom-label-proxy" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516776 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="kube-rbac-proxy" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516783 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4af9a9e9-f2a5-4df9-aea5-7439becee5e1" containerName="registry" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516790 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="alertmanager" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516798 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cd710a5-3058-4286-a8a1-16d798aab259" containerName="console" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.516806 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" containerName="kube-rbac-proxy-web" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:46:20.516980 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140\": container with ID starting with 6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140 not found: ID does not exist" containerID="6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.517004 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140"} err="failed to get container status \"6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140\": rpc error: code = NotFound desc = could not find container \"6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140\": container with ID starting with 6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140 not found: ID does not exist" Jul 30 00:46:20.517012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.517022 2564 scope.go:117] "RemoveContainer" containerID="46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4" Jul 30 00:46:20.517727 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.517326 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4"} err="failed to get container status \"46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4\": rpc error: code = NotFound desc = could not find container \"46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4\": container with ID starting with 46a58af35fdeab54d7822d4c532b7f2745b35d07d1de0114bddbd3730c29caf4 not found: ID does not exist" Jul 30 00:46:20.517727 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.517358 2564 scope.go:117] "RemoveContainer" containerID="d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595" Jul 30 00:46:20.517727 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.517634 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595"} err="failed to get container status \"d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595\": rpc error: code = NotFound desc = could not find container \"d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595\": container with ID starting with d038e33640c24eedad09784dff1db87e756d3aa047b49f036588edd369675595 not found: ID does not exist" Jul 30 00:46:20.517727 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.517656 2564 scope.go:117] "RemoveContainer" containerID="2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655" Jul 30 00:46:20.517907 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.517880 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655"} err="failed to get container status \"2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655\": rpc error: code = NotFound desc = could not find container \"2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655\": container with ID starting with 2f8f2f21f74d7c1a1bf543b2897998ed0e07f37301121c7be180e6a39283f655 not found: ID does not exist" Jul 30 00:46:20.517953 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.517913 2564 scope.go:117] "RemoveContainer" containerID="030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d" Jul 30 00:46:20.518154 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.518138 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d"} err="failed to get container status \"030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d\": rpc error: code = NotFound desc = could not find container \"030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d\": container with ID starting with 030db31799a2cf3377297aa8787d06b14bc37a42469a9994876f4ab8b185de1d not found: ID does not exist" Jul 30 00:46:20.518195 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.518154 2564 scope.go:117] "RemoveContainer" containerID="a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc" Jul 30 00:46:20.518381 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.518363 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc"} err="failed to get container status \"a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc\": rpc error: code = NotFound desc = could not find container \"a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc\": container with ID starting with a3a6dca21d76fafc321ae2f99de25be28ed1d21f53e07469785c13a15abe37cc not found: ID does not exist" Jul 30 00:46:20.518424 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.518382 2564 scope.go:117] "RemoveContainer" containerID="0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c" Jul 30 00:46:20.518599 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.518584 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c"} err="failed to get container status \"0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c\": rpc error: code = NotFound desc = could not find container \"0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c\": container with ID starting with 0a88c3e1cd8e42cf652a459d0f5b3317d0143907898b435de91f26f4fa53934c not found: ID does not exist" Jul 30 00:46:20.518640 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.518599 2564 scope.go:117] "RemoveContainer" containerID="6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140" Jul 30 00:46:20.518819 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.518800 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140"} err="failed to get container status \"6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140\": rpc error: code = NotFound desc = could not find container \"6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140\": container with ID starting with 6fb5eebe89a9910f863115bd05138a6922d8bd2b31cba10d73d3ca6d391e0140 not found: ID does not exist" Jul 30 00:46:20.520457 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.520442 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.522864 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.522846 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 30 00:46:20.523141 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.523124 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 30 00:46:20.523214 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.523124 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4vsfq\"" Jul 30 00:46:20.523410 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.523391 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 30 00:46:20.523491 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.523406 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 30 00:46:20.523491 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.523432 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 30 00:46:20.523491 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.523452 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 30 00:46:20.523491 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.523410 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 30 00:46:20.523823 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.523808 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 30 00:46:20.528137 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.528115 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 30 00:46:20.533345 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.533319 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 00:46:20.661628 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.661589 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/35cd7b71-a14b-4317-b4b0-214f58ad92b5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.661628 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.661635 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.661850 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.661657 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.661850 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.661703 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.661850 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.661745 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.661850 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.661772 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/35cd7b71-a14b-4317-b4b0-214f58ad92b5-config-out\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.661850 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.661789 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35cd7b71-a14b-4317-b4b0-214f58ad92b5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.661850 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.661809 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35cd7b71-a14b-4317-b4b0-214f58ad92b5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.662109 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.661882 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-web-config\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.662109 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.661942 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/35cd7b71-a14b-4317-b4b0-214f58ad92b5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.662109 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.661964 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-config-volume\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.662109 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.661981 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jr4d\" (UniqueName: \"kubernetes.io/projected/35cd7b71-a14b-4317-b4b0-214f58ad92b5-kube-api-access-2jr4d\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.662109 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.662015 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.762606 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.762506 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.762606 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.762559 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.762606 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.762579 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.762606 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.762597 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.762606 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.762617 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/35cd7b71-a14b-4317-b4b0-214f58ad92b5-config-out\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.762980 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.762631 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35cd7b71-a14b-4317-b4b0-214f58ad92b5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.762980 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.762750 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35cd7b71-a14b-4317-b4b0-214f58ad92b5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.762980 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.762830 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-web-config\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.762980 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.762862 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/35cd7b71-a14b-4317-b4b0-214f58ad92b5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.762980 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.762893 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-config-volume\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.762980 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.762944 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jr4d\" (UniqueName: \"kubernetes.io/projected/35cd7b71-a14b-4317-b4b0-214f58ad92b5-kube-api-access-2jr4d\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.763279 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.762981 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.763279 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.763026 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/35cd7b71-a14b-4317-b4b0-214f58ad92b5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.763467 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.763430 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35cd7b71-a14b-4317-b4b0-214f58ad92b5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.763725 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.763706 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/35cd7b71-a14b-4317-b4b0-214f58ad92b5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.763814 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.763790 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35cd7b71-a14b-4317-b4b0-214f58ad92b5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.765707 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.765683 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/35cd7b71-a14b-4317-b4b0-214f58ad92b5-config-out\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.766027 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.765884 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.766027 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.765925 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/35cd7b71-a14b-4317-b4b0-214f58ad92b5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.766173 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.766153 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.766501 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.766471 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.766603 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.766582 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-config-volume\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.766754 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.766740 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.766915 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.766878 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-web-config\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.767597 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.767580 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35cd7b71-a14b-4317-b4b0-214f58ad92b5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.771788 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.771769 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jr4d\" (UniqueName: \"kubernetes.io/projected/35cd7b71-a14b-4317-b4b0-214f58ad92b5-kube-api-access-2jr4d\") pod \"alertmanager-main-0\" (UID: \"35cd7b71-a14b-4317-b4b0-214f58ad92b5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.829859 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.829821 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 30 00:46:20.964213 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:20.964185 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 00:46:20.966400 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:46:20.966369 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35cd7b71_a14b_4317_b4b0_214f58ad92b5.slice/crio-374e9ba0cde2a7bb21641d9635779d6f3ba71cf71c4ae3e786970b5701d7d644 WatchSource:0}: Error finding container 374e9ba0cde2a7bb21641d9635779d6f3ba71cf71c4ae3e786970b5701d7d644: Status 404 returned error can't find the container with id 374e9ba0cde2a7bb21641d9635779d6f3ba71cf71c4ae3e786970b5701d7d644 Jul 30 00:46:21.466078 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:21.466047 2564 generic.go:356] "Generic (PLEG): container finished" podID="35cd7b71-a14b-4317-b4b0-214f58ad92b5" containerID="da5ec957dc686bcf1445d89797ffe95f5fbbaea4801a518a8b5b7fd2781f411c" exitCode=0 Jul 30 00:46:21.466574 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:21.466141 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35cd7b71-a14b-4317-b4b0-214f58ad92b5","Type":"ContainerDied","Data":"da5ec957dc686bcf1445d89797ffe95f5fbbaea4801a518a8b5b7fd2781f411c"} Jul 30 00:46:21.466574 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:21.466229 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35cd7b71-a14b-4317-b4b0-214f58ad92b5","Type":"ContainerStarted","Data":"374e9ba0cde2a7bb21641d9635779d6f3ba71cf71c4ae3e786970b5701d7d644"} Jul 30 00:46:21.667578 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:21.667553 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c2d84b5-73ab-47af-a82a-3f5a4a370234" path="/var/lib/kubelet/pods/1c2d84b5-73ab-47af-a82a-3f5a4a370234/volumes" Jul 30 00:46:22.419193 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.419151 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7c6755f48-r844c"] Jul 30 00:46:22.421627 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.421608 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.423893 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.423867 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 30 00:46:22.424098 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.424079 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 30 00:46:22.424165 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.424153 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 30 00:46:22.424263 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.424243 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 30 00:46:22.424970 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.424949 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 30 00:46:22.425082 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.424984 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-qpp5j\"" Jul 30 00:46:22.428235 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.428215 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 30 00:46:22.434171 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.434151 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c6755f48-r844c"] Jul 30 00:46:22.474185 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.474142 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35cd7b71-a14b-4317-b4b0-214f58ad92b5","Type":"ContainerStarted","Data":"9ef6df83bec749de87d378fc72eea949003d5ca636a31d89d6dbf9be22b7cea6"} Jul 30 00:46:22.474563 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.474210 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35cd7b71-a14b-4317-b4b0-214f58ad92b5","Type":"ContainerStarted","Data":"09337fbc8b4ffeff0eb2c395c6f364ca36796c3b8621e30dc164e8ff5c258e11"} Jul 30 00:46:22.474563 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.474244 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35cd7b71-a14b-4317-b4b0-214f58ad92b5","Type":"ContainerStarted","Data":"4911c8183345fa756e781230a5af5d67d01899fa6647ae4cc15c726e39bbd254"} Jul 30 00:46:22.474563 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.474273 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35cd7b71-a14b-4317-b4b0-214f58ad92b5","Type":"ContainerStarted","Data":"6367cf9b01a5d977383ae4580912f5b6d4c5a69e546e573b163df99f2a14436e"} Jul 30 00:46:22.474563 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.474298 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35cd7b71-a14b-4317-b4b0-214f58ad92b5","Type":"ContainerStarted","Data":"ba8d8953418e8950093ea76cb0a64ba9f4fbc550e48e449f9a51b45e822706ff"} Jul 30 00:46:22.474563 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.474326 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"35cd7b71-a14b-4317-b4b0-214f58ad92b5","Type":"ContainerStarted","Data":"dd2755bb1d3d6b083004944c1ebd2712d6b1bb2275b4b438d0cf8542ebaecd61"} Jul 30 00:46:22.476825 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.476803 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf6t5\" (UniqueName: \"kubernetes.io/projected/56a337c5-cf93-41ce-83cb-d41d9b839600-kube-api-access-rf6t5\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.476865 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.476842 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a337c5-cf93-41ce-83cb-d41d9b839600-serving-certs-ca-bundle\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.476941 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.476928 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/56a337c5-cf93-41ce-83cb-d41d9b839600-secret-telemeter-client\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.476982 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.476959 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56a337c5-cf93-41ce-83cb-d41d9b839600-metrics-client-ca\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.477020 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.476980 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56a337c5-cf93-41ce-83cb-d41d9b839600-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.477063 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.477037 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/56a337c5-cf93-41ce-83cb-d41d9b839600-telemeter-client-tls\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.477063 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.477055 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/56a337c5-cf93-41ce-83cb-d41d9b839600-federate-client-tls\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.477151 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.477070 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a337c5-cf93-41ce-83cb-d41d9b839600-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.497757 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.497706 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.481096714 podStartE2EDuration="2.497690376s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.593662ms" isStatefulPod=false podCreationTimestamp="2026-07-30 00:46:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:46:22.495758862 +0000 UTC m=+185.296715967" watchObservedRunningTime="2026-07-30 00:46:22.497690376 +0000 UTC m=+185.298647494" Jul 30 00:46:22.578486 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.578440 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/56a337c5-cf93-41ce-83cb-d41d9b839600-telemeter-client-tls\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.578649 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.578490 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/56a337c5-cf93-41ce-83cb-d41d9b839600-federate-client-tls\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.578649 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.578543 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a337c5-cf93-41ce-83cb-d41d9b839600-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.578649 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.578639 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf6t5\" (UniqueName: \"kubernetes.io/projected/56a337c5-cf93-41ce-83cb-d41d9b839600-kube-api-access-rf6t5\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.578843 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.578731 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a337c5-cf93-41ce-83cb-d41d9b839600-serving-certs-ca-bundle\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.578843 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.578839 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/56a337c5-cf93-41ce-83cb-d41d9b839600-secret-telemeter-client\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.578997 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.578928 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56a337c5-cf93-41ce-83cb-d41d9b839600-metrics-client-ca\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.578997 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.578965 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56a337c5-cf93-41ce-83cb-d41d9b839600-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.579510 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.579451 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a337c5-cf93-41ce-83cb-d41d9b839600-serving-certs-ca-bundle\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.579622 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.579513 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a337c5-cf93-41ce-83cb-d41d9b839600-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.579871 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.579851 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56a337c5-cf93-41ce-83cb-d41d9b839600-metrics-client-ca\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.581309 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.581276 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/56a337c5-cf93-41ce-83cb-d41d9b839600-telemeter-client-tls\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.581433 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.581416 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/56a337c5-cf93-41ce-83cb-d41d9b839600-federate-client-tls\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.581783 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.581760 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56a337c5-cf93-41ce-83cb-d41d9b839600-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.581822 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.581786 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/56a337c5-cf93-41ce-83cb-d41d9b839600-secret-telemeter-client\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.586746 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.586725 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf6t5\" (UniqueName: \"kubernetes.io/projected/56a337c5-cf93-41ce-83cb-d41d9b839600-kube-api-access-rf6t5\") pod \"telemeter-client-7c6755f48-r844c\" (UID: \"56a337c5-cf93-41ce-83cb-d41d9b839600\") " pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.731706 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.731620 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" Jul 30 00:46:22.865653 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:22.865629 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c6755f48-r844c"] Jul 30 00:46:22.867774 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:46:22.867746 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a337c5_cf93_41ce_83cb_d41d9b839600.slice/crio-e3f2fb25f4fb6d613f6e766fd334ec5d45ffea148e088f892bb305fb29e95a56 WatchSource:0}: Error finding container e3f2fb25f4fb6d613f6e766fd334ec5d45ffea148e088f892bb305fb29e95a56: Status 404 returned error can't find the container with id e3f2fb25f4fb6d613f6e766fd334ec5d45ffea148e088f892bb305fb29e95a56 Jul 30 00:46:23.480126 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:23.480095 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" event={"ID":"56a337c5-cf93-41ce-83cb-d41d9b839600","Type":"ContainerStarted","Data":"e3f2fb25f4fb6d613f6e766fd334ec5d45ffea148e088f892bb305fb29e95a56"} Jul 30 00:46:26.493192 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:26.493159 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" event={"ID":"56a337c5-cf93-41ce-83cb-d41d9b839600","Type":"ContainerStarted","Data":"ee165b516d1db00fcde3b0f6505562dddb966912af1af71f5afc9ab65744f49b"} Jul 30 00:46:26.493602 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:26.493213 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" event={"ID":"56a337c5-cf93-41ce-83cb-d41d9b839600","Type":"ContainerStarted","Data":"f5d63445c6c3a2bfb2226f651ed9427c7055b6159dc3dc2674212d6f121ece35"} Jul 30 00:46:26.493602 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:26.493231 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" event={"ID":"56a337c5-cf93-41ce-83cb-d41d9b839600","Type":"ContainerStarted","Data":"cb61523e02711df4e1b90290ad491b6adf7c24753f3e909db1df7d37cb8501df"} Jul 30 00:46:26.517520 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:26.517472 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7c6755f48-r844c" podStartSLOduration=1.9278243499999999 podStartE2EDuration="4.517457055s" totalImagesPullingTime="2.589632705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:46:26.516650823 +0000 UTC m=+189.317607928" watchObservedRunningTime="2026-07-30 00:46:26.517457055 +0000 UTC m=+189.318414183" Jul 30 00:46:27.290803 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.290767 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-684bf74c6f-9dpkg"] Jul 30 00:46:27.299527 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.299492 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.302449 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.302408 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 30 00:46:27.302654 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.302637 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 30 00:46:27.302884 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.302863 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 30 00:46:27.303092 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.303074 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wjdhw\"" Jul 30 00:46:27.303178 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.303090 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 30 00:46:27.303178 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.303134 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 30 00:46:27.303498 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.303476 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-684bf74c6f-9dpkg"] Jul 30 00:46:27.307983 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.307955 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 30 00:46:27.348725 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.348684 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:27.406681 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.406648 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:27.423975 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.423935 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-trusted-ca-bundle\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.424122 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.423997 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-oauth-config\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.424122 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.424041 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-oauth-serving-cert\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.424122 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.424107 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-serving-cert\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.424243 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.424137 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-service-ca\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.424243 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.424168 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgfcx\" (UniqueName: \"kubernetes.io/projected/d1a0f308-5f52-4745-9ab4-c932a0df99eb-kube-api-access-dgfcx\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.424243 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.424193 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-config\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.496067 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.496027 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:27.524625 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.524587 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-oauth-config\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.524784 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.524638 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-oauth-serving-cert\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.524784 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.524664 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-serving-cert\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.524784 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.524682 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-service-ca\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.524784 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.524705 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgfcx\" (UniqueName: \"kubernetes.io/projected/d1a0f308-5f52-4745-9ab4-c932a0df99eb-kube-api-access-dgfcx\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.524784 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.524731 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-config\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.525059 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.524951 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-trusted-ca-bundle\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.525510 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.525482 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-oauth-serving-cert\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.525612 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.525526 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-service-ca\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.525612 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.525549 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-config\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.525771 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.525758 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-trusted-ca-bundle\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.527286 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.527264 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-serving-cert\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.527387 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.527297 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-oauth-config\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.533861 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.533826 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgfcx\" (UniqueName: \"kubernetes.io/projected/d1a0f308-5f52-4745-9ab4-c932a0df99eb-kube-api-access-dgfcx\") pod \"console-684bf74c6f-9dpkg\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.597257 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.597219 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:27.611215 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.611178 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:27.750446 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:27.750410 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-684bf74c6f-9dpkg"] Jul 30 00:46:27.753638 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:46:27.753609 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1a0f308_5f52_4745_9ab4_c932a0df99eb.slice/crio-08f0f699a6a112e45b4ccd7b7f816baf5b4b0779c86bd2706e723eff8022dba9 WatchSource:0}: Error finding container 08f0f699a6a112e45b4ccd7b7f816baf5b4b0779c86bd2706e723eff8022dba9: Status 404 returned error can't find the container with id 08f0f699a6a112e45b4ccd7b7f816baf5b4b0779c86bd2706e723eff8022dba9 Jul 30 00:46:28.084415 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:28.084384 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:28.186300 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:28.186261 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:28.503677 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:28.503584 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-684bf74c6f-9dpkg" event={"ID":"d1a0f308-5f52-4745-9ab4-c932a0df99eb","Type":"ContainerStarted","Data":"2611dca9a07bd8fb771842a0a85e5e8135ffd933316c31760d9d4c1d0de72036"} Jul 30 00:46:28.503677 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:28.503631 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-684bf74c6f-9dpkg" event={"ID":"d1a0f308-5f52-4745-9ab4-c932a0df99eb","Type":"ContainerStarted","Data":"08f0f699a6a112e45b4ccd7b7f816baf5b4b0779c86bd2706e723eff8022dba9"} Jul 30 00:46:28.520261 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:28.520208 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-684bf74c6f-9dpkg" podStartSLOduration=1.520193041 podStartE2EDuration="1.520193041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:46:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:46:28.518863522 +0000 UTC m=+191.319820629" watchObservedRunningTime="2026-07-30 00:46:28.520193041 +0000 UTC m=+191.321150146" Jul 30 00:46:37.611947 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:37.611883 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:37.612335 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:37.612002 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:37.616612 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:37.616584 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:38.546095 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:38.546068 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:46:38.715487 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:38.715447 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:38.813752 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:38.813658 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:38.860266 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:38.860234 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:50.638775 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:46:50.638734 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:17.565656 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:17.565611 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:17.621741 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:17.621705 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:30.412505 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:30.412455 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:32.559214 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:32.559179 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:32.621587 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:32.621547 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:38.808014 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.807977 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cc6c6ddc4-674b2"] Jul 30 00:47:38.812664 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.812640 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.823335 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.823310 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cc6c6ddc4-674b2"] Jul 30 00:47:38.862474 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.862440 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:38.891480 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.891443 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-trusted-ca-bundle\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.891671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.891490 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-config\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.891671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.891522 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-oauth-config\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.891671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.891617 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdfth\" (UniqueName: \"kubernetes.io/projected/cd73f666-b565-4f74-a928-d1612dd4e6ef-kube-api-access-bdfth\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.891838 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.891694 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-service-ca\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.891838 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.891737 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-oauth-serving-cert\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.891838 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.891782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-serving-cert\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.963849 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.963815 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:38.992274 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.992243 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdfth\" (UniqueName: \"kubernetes.io/projected/cd73f666-b565-4f74-a928-d1612dd4e6ef-kube-api-access-bdfth\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.992433 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.992298 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-service-ca\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.992433 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.992329 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-oauth-serving-cert\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.992433 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.992349 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-serving-cert\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.992433 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.992374 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-trusted-ca-bundle\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.992433 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.992406 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-config\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.992700 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.992536 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-oauth-config\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.993223 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.993192 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-service-ca\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.993223 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.993212 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-oauth-serving-cert\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.993393 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.993202 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-config\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.993486 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.993467 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-trusted-ca-bundle\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.994860 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.994841 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-serving-cert\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.994969 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.994954 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-oauth-config\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:38.999842 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:38.999823 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdfth\" (UniqueName: \"kubernetes.io/projected/cd73f666-b565-4f74-a928-d1612dd4e6ef-kube-api-access-bdfth\") pod \"console-7cc6c6ddc4-674b2\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:39.122542 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:39.122502 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:39.254487 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:39.254462 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cc6c6ddc4-674b2"] Jul 30 00:47:39.256823 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:47:39.256797 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd73f666_b565_4f74_a928_d1612dd4e6ef.slice/crio-1d87330f06166d922e391de849ad4f9b6bde0df2a9d096c012a44fd95847e2fa WatchSource:0}: Error finding container 1d87330f06166d922e391de849ad4f9b6bde0df2a9d096c012a44fd95847e2fa: Status 404 returned error can't find the container with id 1d87330f06166d922e391de849ad4f9b6bde0df2a9d096c012a44fd95847e2fa Jul 30 00:47:39.783321 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:39.783284 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc6c6ddc4-674b2" event={"ID":"cd73f666-b565-4f74-a928-d1612dd4e6ef","Type":"ContainerStarted","Data":"fcc4b28c4606339f89cca00c59fa5af012f402c6a5d30f824d6076359b1736c2"} Jul 30 00:47:39.783510 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:39.783338 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc6c6ddc4-674b2" event={"ID":"cd73f666-b565-4f74-a928-d1612dd4e6ef","Type":"ContainerStarted","Data":"1d87330f06166d922e391de849ad4f9b6bde0df2a9d096c012a44fd95847e2fa"} Jul 30 00:47:39.800757 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:39.800704 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cc6c6ddc4-674b2" podStartSLOduration=1.800684096 podStartE2EDuration="1.800684096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:47:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:47:39.799720734 +0000 UTC m=+262.600677841" watchObservedRunningTime="2026-07-30 00:47:39.800684096 +0000 UTC m=+262.601641202" Jul 30 00:47:44.410666 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:44.410631 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:44.464373 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:44.464326 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:49.123389 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:49.123328 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:49.123825 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:49.123626 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:49.128306 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:49.128279 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:49.825847 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:49.825817 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:47:49.872813 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:49.872775 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-684bf74c6f-9dpkg"] Jul 30 00:47:49.973108 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:49.973068 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:50.041966 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:50.041925 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:50.094344 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:50.094247 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:54.671608 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:54.671571 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:54.772909 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:54.772862 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:55.028804 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:47:55.028706 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:48:14.900098 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:14.900037 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-684bf74c6f-9dpkg" podUID="d1a0f308-5f52-4745-9ab4-c932a0df99eb" containerName="console" containerID="cri-o://2611dca9a07bd8fb771842a0a85e5e8135ffd933316c31760d9d4c1d0de72036" gracePeriod=15 Jul 30 00:48:15.142348 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.142323 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:48:15.184788 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.184703 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-oauth-serving-cert\") pod \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " Jul 30 00:48:15.184788 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.184742 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-trusted-ca-bundle\") pod \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " Jul 30 00:48:15.184788 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.184765 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-config\") pod \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " Jul 30 00:48:15.185088 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.184822 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgfcx\" (UniqueName: \"kubernetes.io/projected/d1a0f308-5f52-4745-9ab4-c932a0df99eb-kube-api-access-dgfcx\") pod \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " Jul 30 00:48:15.185088 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.184889 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-service-ca\") pod \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " Jul 30 00:48:15.185088 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.184986 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-serving-cert\") pod \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " Jul 30 00:48:15.185088 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.185010 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-oauth-config\") pod \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\" (UID: \"d1a0f308-5f52-4745-9ab4-c932a0df99eb\") " Jul 30 00:48:15.185568 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.185200 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d1a0f308-5f52-4745-9ab4-c932a0df99eb" (UID: "d1a0f308-5f52-4745-9ab4-c932a0df99eb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:48:15.185568 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.185264 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d1a0f308-5f52-4745-9ab4-c932a0df99eb" (UID: "d1a0f308-5f52-4745-9ab4-c932a0df99eb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:48:15.185568 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.185293 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-config" (OuterVolumeSpecName: "console-config") pod "d1a0f308-5f52-4745-9ab4-c932a0df99eb" (UID: "d1a0f308-5f52-4745-9ab4-c932a0df99eb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:48:15.185568 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.185374 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-service-ca" (OuterVolumeSpecName: "service-ca") pod "d1a0f308-5f52-4745-9ab4-c932a0df99eb" (UID: "d1a0f308-5f52-4745-9ab4-c932a0df99eb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:48:15.187003 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.186978 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1a0f308-5f52-4745-9ab4-c932a0df99eb-kube-api-access-dgfcx" (OuterVolumeSpecName: "kube-api-access-dgfcx") pod "d1a0f308-5f52-4745-9ab4-c932a0df99eb" (UID: "d1a0f308-5f52-4745-9ab4-c932a0df99eb"). InnerVolumeSpecName "kube-api-access-dgfcx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:48:15.187222 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.187200 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d1a0f308-5f52-4745-9ab4-c932a0df99eb" (UID: "d1a0f308-5f52-4745-9ab4-c932a0df99eb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:48:15.187313 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.187290 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d1a0f308-5f52-4745-9ab4-c932a0df99eb" (UID: "d1a0f308-5f52-4745-9ab4-c932a0df99eb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:48:15.286432 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.286394 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-service-ca\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:48:15.286432 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.286424 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-serving-cert\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:48:15.286432 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.286434 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-oauth-config\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:48:15.286432 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.286443 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-oauth-serving-cert\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:48:15.286688 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.286451 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-trusted-ca-bundle\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:48:15.286688 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.286460 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d1a0f308-5f52-4745-9ab4-c932a0df99eb-console-config\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:48:15.286688 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.286468 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgfcx\" (UniqueName: \"kubernetes.io/projected/d1a0f308-5f52-4745-9ab4-c932a0df99eb-kube-api-access-dgfcx\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:48:15.928970 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.928860 2564 generic.go:356] "Generic (PLEG): container finished" podID="d1a0f308-5f52-4745-9ab4-c932a0df99eb" containerID="2611dca9a07bd8fb771842a0a85e5e8135ffd933316c31760d9d4c1d0de72036" exitCode=0 Jul 30 00:48:15.928970 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.928953 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-684bf74c6f-9dpkg" Jul 30 00:48:15.928970 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.928954 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-684bf74c6f-9dpkg" event={"ID":"d1a0f308-5f52-4745-9ab4-c932a0df99eb","Type":"ContainerDied","Data":"2611dca9a07bd8fb771842a0a85e5e8135ffd933316c31760d9d4c1d0de72036"} Jul 30 00:48:15.929435 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.929014 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-684bf74c6f-9dpkg" event={"ID":"d1a0f308-5f52-4745-9ab4-c932a0df99eb","Type":"ContainerDied","Data":"08f0f699a6a112e45b4ccd7b7f816baf5b4b0779c86bd2706e723eff8022dba9"} Jul 30 00:48:15.929435 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.929038 2564 scope.go:117] "RemoveContainer" containerID="2611dca9a07bd8fb771842a0a85e5e8135ffd933316c31760d9d4c1d0de72036" Jul 30 00:48:15.937366 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.937347 2564 scope.go:117] "RemoveContainer" containerID="2611dca9a07bd8fb771842a0a85e5e8135ffd933316c31760d9d4c1d0de72036" Jul 30 00:48:15.937640 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:48:15.937621 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2611dca9a07bd8fb771842a0a85e5e8135ffd933316c31760d9d4c1d0de72036\": container with ID starting with 2611dca9a07bd8fb771842a0a85e5e8135ffd933316c31760d9d4c1d0de72036 not found: ID does not exist" containerID="2611dca9a07bd8fb771842a0a85e5e8135ffd933316c31760d9d4c1d0de72036" Jul 30 00:48:15.937688 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.937648 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2611dca9a07bd8fb771842a0a85e5e8135ffd933316c31760d9d4c1d0de72036"} err="failed to get container status \"2611dca9a07bd8fb771842a0a85e5e8135ffd933316c31760d9d4c1d0de72036\": rpc error: code = NotFound desc = could not find container \"2611dca9a07bd8fb771842a0a85e5e8135ffd933316c31760d9d4c1d0de72036\": container with ID starting with 2611dca9a07bd8fb771842a0a85e5e8135ffd933316c31760d9d4c1d0de72036 not found: ID does not exist" Jul 30 00:48:15.945079 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.945056 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-684bf74c6f-9dpkg"] Jul 30 00:48:15.948183 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:15.948151 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-684bf74c6f-9dpkg"] Jul 30 00:48:17.540422 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:17.540391 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 30 00:48:17.667653 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:17.667480 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1a0f308-5f52-4745-9ab4-c932a0df99eb" path="/var/lib/kubelet/pods/d1a0f308-5f52-4745-9ab4-c932a0df99eb/volumes" Jul 30 00:48:46.438521 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:46.438484 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:48:47.673179 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.673146 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk"] Jul 30 00:48:47.673641 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.673592 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1a0f308-5f52-4745-9ab4-c932a0df99eb" containerName="console" Jul 30 00:48:47.673641 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.673611 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1a0f308-5f52-4745-9ab4-c932a0df99eb" containerName="console" Jul 30 00:48:47.673753 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.673678 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1a0f308-5f52-4745-9ab4-c932a0df99eb" containerName="console" Jul 30 00:48:47.676875 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.676843 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk"] Jul 30 00:48:47.677026 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.676980 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" Jul 30 00:48:47.679585 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.679557 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 30 00:48:47.679702 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.679589 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-55x9b\"" Jul 30 00:48:47.679702 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.679578 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 30 00:48:47.768509 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.768470 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fc99378-5f9a-49fb-a187-75897f0aa2e0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk\" (UID: \"1fc99378-5f9a-49fb-a187-75897f0aa2e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" Jul 30 00:48:47.768509 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.768507 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fc99378-5f9a-49fb-a187-75897f0aa2e0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk\" (UID: \"1fc99378-5f9a-49fb-a187-75897f0aa2e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" Jul 30 00:48:47.768729 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.768536 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4zh7\" (UniqueName: \"kubernetes.io/projected/1fc99378-5f9a-49fb-a187-75897f0aa2e0-kube-api-access-k4zh7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk\" (UID: \"1fc99378-5f9a-49fb-a187-75897f0aa2e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" Jul 30 00:48:47.869948 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.869874 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fc99378-5f9a-49fb-a187-75897f0aa2e0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk\" (UID: \"1fc99378-5f9a-49fb-a187-75897f0aa2e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" Jul 30 00:48:47.870143 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.869961 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fc99378-5f9a-49fb-a187-75897f0aa2e0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk\" (UID: \"1fc99378-5f9a-49fb-a187-75897f0aa2e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" Jul 30 00:48:47.870143 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.870003 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4zh7\" (UniqueName: \"kubernetes.io/projected/1fc99378-5f9a-49fb-a187-75897f0aa2e0-kube-api-access-k4zh7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk\" (UID: \"1fc99378-5f9a-49fb-a187-75897f0aa2e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" Jul 30 00:48:47.870255 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.870246 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fc99378-5f9a-49fb-a187-75897f0aa2e0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk\" (UID: \"1fc99378-5f9a-49fb-a187-75897f0aa2e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" Jul 30 00:48:47.870305 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.870280 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fc99378-5f9a-49fb-a187-75897f0aa2e0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk\" (UID: \"1fc99378-5f9a-49fb-a187-75897f0aa2e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" Jul 30 00:48:47.878226 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.878194 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4zh7\" (UniqueName: \"kubernetes.io/projected/1fc99378-5f9a-49fb-a187-75897f0aa2e0-kube-api-access-k4zh7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk\" (UID: \"1fc99378-5f9a-49fb-a187-75897f0aa2e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" Jul 30 00:48:47.987227 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:47.987137 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" Jul 30 00:48:48.114823 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:48.114787 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk"] Jul 30 00:48:48.117694 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:48:48.117670 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc99378_5f9a_49fb_a187_75897f0aa2e0.slice/crio-731a0712134961794b65c7ab6f3db473a608c2dca0dda7afa112d746d63a3c23 WatchSource:0}: Error finding container 731a0712134961794b65c7ab6f3db473a608c2dca0dda7afa112d746d63a3c23: Status 404 returned error can't find the container with id 731a0712134961794b65c7ab6f3db473a608c2dca0dda7afa112d746d63a3c23 Jul 30 00:48:48.119730 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:48.119711 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 00:48:49.058187 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:49.058153 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" event={"ID":"1fc99378-5f9a-49fb-a187-75897f0aa2e0","Type":"ContainerStarted","Data":"731a0712134961794b65c7ab6f3db473a608c2dca0dda7afa112d746d63a3c23"} Jul 30 00:48:49.144370 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:49.144327 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:48:56.089069 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:56.089035 2564 generic.go:356] "Generic (PLEG): container finished" podID="1fc99378-5f9a-49fb-a187-75897f0aa2e0" containerID="2be2ce2b95a5590ab17f0a40e888faee70744171822e0260ee2eddb2464b51a8" exitCode=0 Jul 30 00:48:56.089535 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:56.089088 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" event={"ID":"1fc99378-5f9a-49fb-a187-75897f0aa2e0","Type":"ContainerDied","Data":"2be2ce2b95a5590ab17f0a40e888faee70744171822e0260ee2eddb2464b51a8"} Jul 30 00:48:59.105440 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:59.105405 2564 generic.go:356] "Generic (PLEG): container finished" podID="1fc99378-5f9a-49fb-a187-75897f0aa2e0" containerID="d45a5fa661624d73c467a8218ef0c72ead8badb39262315437ef15ce3278621f" exitCode=0 Jul 30 00:48:59.105807 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:48:59.105446 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" event={"ID":"1fc99378-5f9a-49fb-a187-75897f0aa2e0","Type":"ContainerDied","Data":"d45a5fa661624d73c467a8218ef0c72ead8badb39262315437ef15ce3278621f"} Jul 30 00:49:06.140053 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:06.140011 2564 generic.go:356] "Generic (PLEG): container finished" podID="1fc99378-5f9a-49fb-a187-75897f0aa2e0" containerID="a50d5d21ad2ce285540ef3b109ecb196ba40e3f3091161365968666c6dce8dce" exitCode=0 Jul 30 00:49:06.140503 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:06.140094 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" event={"ID":"1fc99378-5f9a-49fb-a187-75897f0aa2e0","Type":"ContainerDied","Data":"a50d5d21ad2ce285540ef3b109ecb196ba40e3f3091161365968666c6dce8dce"} Jul 30 00:49:07.303359 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:07.303321 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" Jul 30 00:49:07.448080 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:07.447985 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4zh7\" (UniqueName: \"kubernetes.io/projected/1fc99378-5f9a-49fb-a187-75897f0aa2e0-kube-api-access-k4zh7\") pod \"1fc99378-5f9a-49fb-a187-75897f0aa2e0\" (UID: \"1fc99378-5f9a-49fb-a187-75897f0aa2e0\") " Jul 30 00:49:07.448278 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:07.448095 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fc99378-5f9a-49fb-a187-75897f0aa2e0-bundle\") pod \"1fc99378-5f9a-49fb-a187-75897f0aa2e0\" (UID: \"1fc99378-5f9a-49fb-a187-75897f0aa2e0\") " Jul 30 00:49:07.448278 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:07.448153 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fc99378-5f9a-49fb-a187-75897f0aa2e0-util\") pod \"1fc99378-5f9a-49fb-a187-75897f0aa2e0\" (UID: \"1fc99378-5f9a-49fb-a187-75897f0aa2e0\") " Jul 30 00:49:07.448762 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:07.448737 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fc99378-5f9a-49fb-a187-75897f0aa2e0-bundle" (OuterVolumeSpecName: "bundle") pod "1fc99378-5f9a-49fb-a187-75897f0aa2e0" (UID: "1fc99378-5f9a-49fb-a187-75897f0aa2e0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:49:07.450243 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:07.450217 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fc99378-5f9a-49fb-a187-75897f0aa2e0-kube-api-access-k4zh7" (OuterVolumeSpecName: "kube-api-access-k4zh7") pod "1fc99378-5f9a-49fb-a187-75897f0aa2e0" (UID: "1fc99378-5f9a-49fb-a187-75897f0aa2e0"). InnerVolumeSpecName "kube-api-access-k4zh7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:49:07.452278 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:07.452254 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fc99378-5f9a-49fb-a187-75897f0aa2e0-util" (OuterVolumeSpecName: "util") pod "1fc99378-5f9a-49fb-a187-75897f0aa2e0" (UID: "1fc99378-5f9a-49fb-a187-75897f0aa2e0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:49:07.548743 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:07.548705 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fc99378-5f9a-49fb-a187-75897f0aa2e0-bundle\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:49:07.548743 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:07.548736 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fc99378-5f9a-49fb-a187-75897f0aa2e0-util\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:49:07.548743 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:07.548749 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4zh7\" (UniqueName: \"kubernetes.io/projected/1fc99378-5f9a-49fb-a187-75897f0aa2e0-kube-api-access-k4zh7\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:49:08.149561 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:08.149532 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" Jul 30 00:49:08.149561 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:08.149545 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hbtlk" event={"ID":"1fc99378-5f9a-49fb-a187-75897f0aa2e0","Type":"ContainerDied","Data":"731a0712134961794b65c7ab6f3db473a608c2dca0dda7afa112d746d63a3c23"} Jul 30 00:49:08.149776 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:08.149588 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="731a0712134961794b65c7ab6f3db473a608c2dca0dda7afa112d746d63a3c23" Jul 30 00:49:13.677332 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:13.677236 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:14.458814 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:14.458770 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:15.581796 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.581752 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-655xs"] Jul 30 00:49:15.582278 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.582230 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fc99378-5f9a-49fb-a187-75897f0aa2e0" containerName="util" Jul 30 00:49:15.582278 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.582249 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fc99378-5f9a-49fb-a187-75897f0aa2e0" containerName="util" Jul 30 00:49:15.582278 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.582268 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fc99378-5f9a-49fb-a187-75897f0aa2e0" containerName="pull" Jul 30 00:49:15.582278 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.582276 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fc99378-5f9a-49fb-a187-75897f0aa2e0" containerName="pull" Jul 30 00:49:15.582478 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.582303 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fc99378-5f9a-49fb-a187-75897f0aa2e0" containerName="extract" Jul 30 00:49:15.582478 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.582312 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fc99378-5f9a-49fb-a187-75897f0aa2e0" containerName="extract" Jul 30 00:49:15.582478 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.582396 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fc99378-5f9a-49fb-a187-75897f0aa2e0" containerName="extract" Jul 30 00:49:15.585400 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.585379 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-655xs" Jul 30 00:49:15.587661 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.587637 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 30 00:49:15.587784 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.587662 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-55pgd\"" Jul 30 00:49:15.587784 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.587700 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 30 00:49:15.597585 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.597555 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-655xs"] Jul 30 00:49:15.726282 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.726238 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l65v5\" (UniqueName: \"kubernetes.io/projected/30641ae3-b556-4fdf-84ee-98821520ed42-kube-api-access-l65v5\") pod \"cert-manager-operator-controller-manager-55c444bf86-655xs\" (UID: \"30641ae3-b556-4fdf-84ee-98821520ed42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-655xs" Jul 30 00:49:15.726282 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.726286 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30641ae3-b556-4fdf-84ee-98821520ed42-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-655xs\" (UID: \"30641ae3-b556-4fdf-84ee-98821520ed42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-655xs" Jul 30 00:49:15.827670 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.827629 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30641ae3-b556-4fdf-84ee-98821520ed42-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-655xs\" (UID: \"30641ae3-b556-4fdf-84ee-98821520ed42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-655xs" Jul 30 00:49:15.827851 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.827709 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l65v5\" (UniqueName: \"kubernetes.io/projected/30641ae3-b556-4fdf-84ee-98821520ed42-kube-api-access-l65v5\") pod \"cert-manager-operator-controller-manager-55c444bf86-655xs\" (UID: \"30641ae3-b556-4fdf-84ee-98821520ed42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-655xs" Jul 30 00:49:15.828046 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.828028 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30641ae3-b556-4fdf-84ee-98821520ed42-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-655xs\" (UID: \"30641ae3-b556-4fdf-84ee-98821520ed42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-655xs" Jul 30 00:49:15.836045 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.835981 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l65v5\" (UniqueName: \"kubernetes.io/projected/30641ae3-b556-4fdf-84ee-98821520ed42-kube-api-access-l65v5\") pod \"cert-manager-operator-controller-manager-55c444bf86-655xs\" (UID: \"30641ae3-b556-4fdf-84ee-98821520ed42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-655xs" Jul 30 00:49:15.894863 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:15.894828 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-655xs" Jul 30 00:49:16.044773 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:16.044746 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-655xs"] Jul 30 00:49:16.047296 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:49:16.047257 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30641ae3_b556_4fdf_84ee_98821520ed42.slice/crio-d579a73b82967d4b2bf56ab404031257cdfa03dd63b8f981c59e5b183231c0b5 WatchSource:0}: Error finding container d579a73b82967d4b2bf56ab404031257cdfa03dd63b8f981c59e5b183231c0b5: Status 404 returned error can't find the container with id d579a73b82967d4b2bf56ab404031257cdfa03dd63b8f981c59e5b183231c0b5 Jul 30 00:49:16.188833 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:16.188752 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-655xs" event={"ID":"30641ae3-b556-4fdf-84ee-98821520ed42","Type":"ContainerStarted","Data":"d579a73b82967d4b2bf56ab404031257cdfa03dd63b8f981c59e5b183231c0b5"} Jul 30 00:49:17.285708 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:17.285652 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:17.387122 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:17.387088 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:18.200469 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:18.200428 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-655xs" event={"ID":"30641ae3-b556-4fdf-84ee-98821520ed42","Type":"ContainerStarted","Data":"2e267a404ed5c5c35ddce94aafabcb5f44207fc0500bd3ac6e171facca3c3267"} Jul 30 00:49:18.221797 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:18.221741 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-655xs" podStartSLOduration=1.280414935 podStartE2EDuration="3.221723298s" totalImagesPullingTime="1.941308363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:49:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:49:18.219981788 +0000 UTC m=+361.020938884" watchObservedRunningTime="2026-07-30 00:49:18.221723298 +0000 UTC m=+361.022680404" Jul 30 00:49:18.550598 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:18.550549 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:18.643022 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:18.642981 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:19.316112 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.316076 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7"] Jul 30 00:49:19.319725 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.319708 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" Jul 30 00:49:19.322130 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.322104 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 30 00:49:19.322258 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.322240 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 30 00:49:19.323064 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.323044 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-55x9b\"" Jul 30 00:49:19.328724 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.328699 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7"] Jul 30 00:49:19.461703 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.461662 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3f40267-429b-47f8-874f-f9d4937883f5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7\" (UID: \"b3f40267-429b-47f8-874f-f9d4937883f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" Jul 30 00:49:19.461703 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.461702 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8swkq\" (UniqueName: \"kubernetes.io/projected/b3f40267-429b-47f8-874f-f9d4937883f5-kube-api-access-8swkq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7\" (UID: \"b3f40267-429b-47f8-874f-f9d4937883f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" Jul 30 00:49:19.461991 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.461833 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3f40267-429b-47f8-874f-f9d4937883f5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7\" (UID: \"b3f40267-429b-47f8-874f-f9d4937883f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" Jul 30 00:49:19.563359 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.563321 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3f40267-429b-47f8-874f-f9d4937883f5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7\" (UID: \"b3f40267-429b-47f8-874f-f9d4937883f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" Jul 30 00:49:19.563737 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.563385 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3f40267-429b-47f8-874f-f9d4937883f5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7\" (UID: \"b3f40267-429b-47f8-874f-f9d4937883f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" Jul 30 00:49:19.563737 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.563411 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8swkq\" (UniqueName: \"kubernetes.io/projected/b3f40267-429b-47f8-874f-f9d4937883f5-kube-api-access-8swkq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7\" (UID: \"b3f40267-429b-47f8-874f-f9d4937883f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" Jul 30 00:49:19.563737 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.563704 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3f40267-429b-47f8-874f-f9d4937883f5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7\" (UID: \"b3f40267-429b-47f8-874f-f9d4937883f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" Jul 30 00:49:19.563878 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.563754 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3f40267-429b-47f8-874f-f9d4937883f5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7\" (UID: \"b3f40267-429b-47f8-874f-f9d4937883f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" Jul 30 00:49:19.574258 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.574225 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8swkq\" (UniqueName: \"kubernetes.io/projected/b3f40267-429b-47f8-874f-f9d4937883f5-kube-api-access-8swkq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7\" (UID: \"b3f40267-429b-47f8-874f-f9d4937883f5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" Jul 30 00:49:19.629392 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.629357 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" Jul 30 00:49:19.651618 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.651574 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:19.757361 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.757325 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:19.768182 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:19.768154 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7"] Jul 30 00:49:19.771864 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:49:19.771828 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3f40267_429b_47f8_874f_f9d4937883f5.slice/crio-6b048123c5c39d703090579806c0e687838b463f76206a7146df72e1fdc3bcb1 WatchSource:0}: Error finding container 6b048123c5c39d703090579806c0e687838b463f76206a7146df72e1fdc3bcb1: Status 404 returned error can't find the container with id 6b048123c5c39d703090579806c0e687838b463f76206a7146df72e1fdc3bcb1 Jul 30 00:49:20.210933 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:20.210832 2564 generic.go:356] "Generic (PLEG): container finished" podID="b3f40267-429b-47f8-874f-f9d4937883f5" containerID="8ff0c1d2d56a59ae41af8503736183f90c26b0d9f07d6c635e7f2c7f29dd8265" exitCode=0 Jul 30 00:49:20.210933 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:20.210881 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" event={"ID":"b3f40267-429b-47f8-874f-f9d4937883f5","Type":"ContainerDied","Data":"8ff0c1d2d56a59ae41af8503736183f90c26b0d9f07d6c635e7f2c7f29dd8265"} Jul 30 00:49:20.211127 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:20.210941 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" event={"ID":"b3f40267-429b-47f8-874f-f9d4937883f5","Type":"ContainerStarted","Data":"6b048123c5c39d703090579806c0e687838b463f76206a7146df72e1fdc3bcb1"} Jul 30 00:49:21.462258 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:21.462215 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:22.053542 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:22.053509 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:22.141602 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:22.141561 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ccpb6"] Jul 30 00:49:22.152274 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:22.152226 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ccpb6" Jul 30 00:49:22.154662 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:22.154631 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 30 00:49:22.154662 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:22.154631 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 30 00:49:22.155376 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:22.155356 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-mpsbp\"" Jul 30 00:49:22.156415 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:22.156176 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ccpb6"] Jul 30 00:49:22.288135 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:22.288098 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4lms\" (UniqueName: \"kubernetes.io/projected/1ed808ac-dbce-42f3-ab6a-613e0e1e92a0-kube-api-access-p4lms\") pod \"cert-manager-webhook-6486ffd997-ccpb6\" (UID: \"1ed808ac-dbce-42f3-ab6a-613e0e1e92a0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ccpb6" Jul 30 00:49:22.288297 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:22.288154 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ed808ac-dbce-42f3-ab6a-613e0e1e92a0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ccpb6\" (UID: \"1ed808ac-dbce-42f3-ab6a-613e0e1e92a0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ccpb6" Jul 30 00:49:22.389512 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:22.389479 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4lms\" (UniqueName: \"kubernetes.io/projected/1ed808ac-dbce-42f3-ab6a-613e0e1e92a0-kube-api-access-p4lms\") pod \"cert-manager-webhook-6486ffd997-ccpb6\" (UID: \"1ed808ac-dbce-42f3-ab6a-613e0e1e92a0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ccpb6" Jul 30 00:49:22.389743 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:22.389527 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ed808ac-dbce-42f3-ab6a-613e0e1e92a0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ccpb6\" (UID: \"1ed808ac-dbce-42f3-ab6a-613e0e1e92a0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ccpb6" Jul 30 00:49:22.397610 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:22.397572 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ed808ac-dbce-42f3-ab6a-613e0e1e92a0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ccpb6\" (UID: \"1ed808ac-dbce-42f3-ab6a-613e0e1e92a0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ccpb6" Jul 30 00:49:22.397733 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:22.397712 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4lms\" (UniqueName: \"kubernetes.io/projected/1ed808ac-dbce-42f3-ab6a-613e0e1e92a0-kube-api-access-p4lms\") pod \"cert-manager-webhook-6486ffd997-ccpb6\" (UID: \"1ed808ac-dbce-42f3-ab6a-613e0e1e92a0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ccpb6" Jul 30 00:49:22.474008 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:22.473978 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ccpb6" Jul 30 00:49:22.610727 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:22.610693 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ccpb6"] Jul 30 00:49:22.613698 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:49:22.613667 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ed808ac_dbce_42f3_ab6a_613e0e1e92a0.slice/crio-60e75606838305f8a55c4c4110ebce7b436dca48fa96770f5b9248bcc4f7d98a WatchSource:0}: Error finding container 60e75606838305f8a55c4c4110ebce7b436dca48fa96770f5b9248bcc4f7d98a: Status 404 returned error can't find the container with id 60e75606838305f8a55c4c4110ebce7b436dca48fa96770f5b9248bcc4f7d98a Jul 30 00:49:23.227755 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:23.227718 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ccpb6" event={"ID":"1ed808ac-dbce-42f3-ab6a-613e0e1e92a0","Type":"ContainerStarted","Data":"60e75606838305f8a55c4c4110ebce7b436dca48fa96770f5b9248bcc4f7d98a"} Jul 30 00:49:23.229356 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:23.229336 2564 generic.go:356] "Generic (PLEG): container finished" podID="b3f40267-429b-47f8-874f-f9d4937883f5" containerID="2011d35f18d6141f878b0e9bbc2ace2ca5de291cfc7b6556f779c5f99d7779c3" exitCode=0 Jul 30 00:49:23.229449 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:23.229366 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" event={"ID":"b3f40267-429b-47f8-874f-f9d4937883f5","Type":"ContainerDied","Data":"2011d35f18d6141f878b0e9bbc2ace2ca5de291cfc7b6556f779c5f99d7779c3"} Jul 30 00:49:24.236640 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:24.236604 2564 generic.go:356] "Generic (PLEG): container finished" podID="b3f40267-429b-47f8-874f-f9d4937883f5" containerID="b6134468e94bec4df2dbede8c3f6907495d96dc0d9c18b957fbafc0368d38d10" exitCode=0 Jul 30 00:49:24.237067 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:24.236670 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" event={"ID":"b3f40267-429b-47f8-874f-f9d4937883f5","Type":"ContainerDied","Data":"b6134468e94bec4df2dbede8c3f6907495d96dc0d9c18b957fbafc0368d38d10"} Jul 30 00:49:25.337128 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:25.337094 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:25.391845 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:25.391797 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:25.472857 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:25.472829 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" Jul 30 00:49:25.619025 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:25.618991 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3f40267-429b-47f8-874f-f9d4937883f5-util\") pod \"b3f40267-429b-47f8-874f-f9d4937883f5\" (UID: \"b3f40267-429b-47f8-874f-f9d4937883f5\") " Jul 30 00:49:25.619220 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:25.619124 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3f40267-429b-47f8-874f-f9d4937883f5-bundle\") pod \"b3f40267-429b-47f8-874f-f9d4937883f5\" (UID: \"b3f40267-429b-47f8-874f-f9d4937883f5\") " Jul 30 00:49:25.619220 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:25.619157 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8swkq\" (UniqueName: \"kubernetes.io/projected/b3f40267-429b-47f8-874f-f9d4937883f5-kube-api-access-8swkq\") pod \"b3f40267-429b-47f8-874f-f9d4937883f5\" (UID: \"b3f40267-429b-47f8-874f-f9d4937883f5\") " Jul 30 00:49:25.619505 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:25.619475 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f40267-429b-47f8-874f-f9d4937883f5-bundle" (OuterVolumeSpecName: "bundle") pod "b3f40267-429b-47f8-874f-f9d4937883f5" (UID: "b3f40267-429b-47f8-874f-f9d4937883f5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:49:25.621701 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:25.621673 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3f40267-429b-47f8-874f-f9d4937883f5-kube-api-access-8swkq" (OuterVolumeSpecName: "kube-api-access-8swkq") pod "b3f40267-429b-47f8-874f-f9d4937883f5" (UID: "b3f40267-429b-47f8-874f-f9d4937883f5"). InnerVolumeSpecName "kube-api-access-8swkq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:49:25.625374 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:25.625320 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f40267-429b-47f8-874f-f9d4937883f5-util" (OuterVolumeSpecName: "util") pod "b3f40267-429b-47f8-874f-f9d4937883f5" (UID: "b3f40267-429b-47f8-874f-f9d4937883f5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:49:25.719927 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:25.719858 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3f40267-429b-47f8-874f-f9d4937883f5-util\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:49:25.719927 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:25.719920 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3f40267-429b-47f8-874f-f9d4937883f5-bundle\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:49:25.719927 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:25.719933 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8swkq\" (UniqueName: \"kubernetes.io/projected/b3f40267-429b-47f8-874f-f9d4937883f5-kube-api-access-8swkq\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:49:26.248145 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:26.248112 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ccpb6" event={"ID":"1ed808ac-dbce-42f3-ab6a-613e0e1e92a0","Type":"ContainerStarted","Data":"0381cff2ffa04cce7521588044015a751c14f5fe89dee9c45f4663c6453a4030"} Jul 30 00:49:26.248371 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:26.248205 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ccpb6" Jul 30 00:49:26.249752 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:26.249734 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" Jul 30 00:49:26.249845 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:26.249733 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgblz7" event={"ID":"b3f40267-429b-47f8-874f-f9d4937883f5","Type":"ContainerDied","Data":"6b048123c5c39d703090579806c0e687838b463f76206a7146df72e1fdc3bcb1"} Jul 30 00:49:26.249884 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:26.249844 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b048123c5c39d703090579806c0e687838b463f76206a7146df72e1fdc3bcb1" Jul 30 00:49:26.264402 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:26.264349 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-ccpb6" podStartSLOduration=1.345908747 podStartE2EDuration="4.264333072s" totalImagesPullingTime="2.918424325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:49:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:49:26.263465066 +0000 UTC m=+369.064422172" watchObservedRunningTime="2026-07-30 00:49:26.264333072 +0000 UTC m=+369.065290178" Jul 30 00:49:26.729192 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:26.729155 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:26.783012 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:26.782979 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:30.094489 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:30.094447 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:30.194216 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:30.194182 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:30.859798 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:30.859768 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:31.013211 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.013177 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ctxq2"] Jul 30 00:49:31.013534 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.013518 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3f40267-429b-47f8-874f-f9d4937883f5" containerName="extract" Jul 30 00:49:31.013534 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.013535 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3f40267-429b-47f8-874f-f9d4937883f5" containerName="extract" Jul 30 00:49:31.013656 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.013551 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3f40267-429b-47f8-874f-f9d4937883f5" containerName="util" Jul 30 00:49:31.013656 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.013557 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3f40267-429b-47f8-874f-f9d4937883f5" containerName="util" Jul 30 00:49:31.013656 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.013574 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3f40267-429b-47f8-874f-f9d4937883f5" containerName="pull" Jul 30 00:49:31.013656 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.013579 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3f40267-429b-47f8-874f-f9d4937883f5" containerName="pull" Jul 30 00:49:31.013656 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.013637 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3f40267-429b-47f8-874f-f9d4937883f5" containerName="extract" Jul 30 00:49:31.018283 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.018261 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ctxq2" Jul 30 00:49:31.020558 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.020536 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-x86sp\"" Jul 30 00:49:31.029679 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.029650 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ctxq2"] Jul 30 00:49:31.061649 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.061605 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9c533a2e-6947-4269-addd-9a084cd74ce5-bound-sa-token\") pod \"cert-manager-55b4d954d5-ctxq2\" (UID: \"9c533a2e-6947-4269-addd-9a084cd74ce5\") " pod="cert-manager/cert-manager-55b4d954d5-ctxq2" Jul 30 00:49:31.061798 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.061695 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77g7s\" (UniqueName: \"kubernetes.io/projected/9c533a2e-6947-4269-addd-9a084cd74ce5-kube-api-access-77g7s\") pod \"cert-manager-55b4d954d5-ctxq2\" (UID: \"9c533a2e-6947-4269-addd-9a084cd74ce5\") " pod="cert-manager/cert-manager-55b4d954d5-ctxq2" Jul 30 00:49:31.162528 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.162441 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9c533a2e-6947-4269-addd-9a084cd74ce5-bound-sa-token\") pod \"cert-manager-55b4d954d5-ctxq2\" (UID: \"9c533a2e-6947-4269-addd-9a084cd74ce5\") " pod="cert-manager/cert-manager-55b4d954d5-ctxq2" Jul 30 00:49:31.162528 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.162490 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77g7s\" (UniqueName: \"kubernetes.io/projected/9c533a2e-6947-4269-addd-9a084cd74ce5-kube-api-access-77g7s\") pod \"cert-manager-55b4d954d5-ctxq2\" (UID: \"9c533a2e-6947-4269-addd-9a084cd74ce5\") " pod="cert-manager/cert-manager-55b4d954d5-ctxq2" Jul 30 00:49:31.170737 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.170713 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9c533a2e-6947-4269-addd-9a084cd74ce5-bound-sa-token\") pod \"cert-manager-55b4d954d5-ctxq2\" (UID: \"9c533a2e-6947-4269-addd-9a084cd74ce5\") " pod="cert-manager/cert-manager-55b4d954d5-ctxq2" Jul 30 00:49:31.171045 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.171022 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77g7s\" (UniqueName: \"kubernetes.io/projected/9c533a2e-6947-4269-addd-9a084cd74ce5-kube-api-access-77g7s\") pod \"cert-manager-55b4d954d5-ctxq2\" (UID: \"9c533a2e-6947-4269-addd-9a084cd74ce5\") " pod="cert-manager/cert-manager-55b4d954d5-ctxq2" Jul 30 00:49:31.333300 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.333262 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ctxq2" Jul 30 00:49:31.469152 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:31.469121 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ctxq2"] Jul 30 00:49:31.471685 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:49:31.471659 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c533a2e_6947_4269_addd_9a084cd74ce5.slice/crio-9eb2cd187573f62c96fcc93f83af3adc23a2843e8e612dfbacf4a558ce7743fa WatchSource:0}: Error finding container 9eb2cd187573f62c96fcc93f83af3adc23a2843e8e612dfbacf4a558ce7743fa: Status 404 returned error can't find the container with id 9eb2cd187573f62c96fcc93f83af3adc23a2843e8e612dfbacf4a558ce7743fa Jul 30 00:49:32.256876 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:32.256847 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ccpb6" Jul 30 00:49:32.277585 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:32.277552 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ctxq2" event={"ID":"9c533a2e-6947-4269-addd-9a084cd74ce5","Type":"ContainerStarted","Data":"fbbe96b8806e63104b8c1bc3a909e93b6a715dc971f01356c144cc4556b98793"} Jul 30 00:49:32.277775 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:32.277600 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ctxq2" event={"ID":"9c533a2e-6947-4269-addd-9a084cd74ce5","Type":"ContainerStarted","Data":"9eb2cd187573f62c96fcc93f83af3adc23a2843e8e612dfbacf4a558ce7743fa"} Jul 30 00:49:32.295787 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:32.295725 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-ctxq2" podStartSLOduration=2.295705354 podStartE2EDuration="2.295705354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:49:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:49:32.294475044 +0000 UTC m=+375.095432152" watchObservedRunningTime="2026-07-30 00:49:32.295705354 +0000 UTC m=+375.096662460" Jul 30 00:49:36.511969 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:36.511929 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:36.612210 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:36.612158 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:38.566583 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:38.566536 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:41.155701 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.155667 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd"] Jul 30 00:49:41.159055 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.159038 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" Jul 30 00:49:41.161254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.161224 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 30 00:49:41.161358 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.161265 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 30 00:49:41.162105 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.162081 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-55x9b\"" Jul 30 00:49:41.168323 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.168175 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd"] Jul 30 00:49:41.239053 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.239010 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a335e848-73cc-4844-8007-2bb942c21949-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd\" (UID: \"a335e848-73cc-4844-8007-2bb942c21949\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" Jul 30 00:49:41.239053 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.239068 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js7n9\" (UniqueName: \"kubernetes.io/projected/a335e848-73cc-4844-8007-2bb942c21949-kube-api-access-js7n9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd\" (UID: \"a335e848-73cc-4844-8007-2bb942c21949\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" Jul 30 00:49:41.239292 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.239171 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a335e848-73cc-4844-8007-2bb942c21949-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd\" (UID: \"a335e848-73cc-4844-8007-2bb942c21949\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" Jul 30 00:49:41.340165 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.340130 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js7n9\" (UniqueName: \"kubernetes.io/projected/a335e848-73cc-4844-8007-2bb942c21949-kube-api-access-js7n9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd\" (UID: \"a335e848-73cc-4844-8007-2bb942c21949\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" Jul 30 00:49:41.340358 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.340196 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a335e848-73cc-4844-8007-2bb942c21949-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd\" (UID: \"a335e848-73cc-4844-8007-2bb942c21949\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" Jul 30 00:49:41.340358 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.340231 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a335e848-73cc-4844-8007-2bb942c21949-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd\" (UID: \"a335e848-73cc-4844-8007-2bb942c21949\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" Jul 30 00:49:41.340609 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.340593 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a335e848-73cc-4844-8007-2bb942c21949-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd\" (UID: \"a335e848-73cc-4844-8007-2bb942c21949\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" Jul 30 00:49:41.340657 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.340615 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a335e848-73cc-4844-8007-2bb942c21949-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd\" (UID: \"a335e848-73cc-4844-8007-2bb942c21949\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" Jul 30 00:49:41.348001 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.347965 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js7n9\" (UniqueName: \"kubernetes.io/projected/a335e848-73cc-4844-8007-2bb942c21949-kube-api-access-js7n9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd\" (UID: \"a335e848-73cc-4844-8007-2bb942c21949\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" Jul 30 00:49:41.474240 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.474158 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" Jul 30 00:49:41.602988 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:41.602888 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd"] Jul 30 00:49:41.605461 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:49:41.605433 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda335e848_73cc_4844_8007_2bb942c21949.slice/crio-fabaa23439d4e8af5109663a402eb11dc5c8fcefaa3e52f0d2d7db29038e1cb5 WatchSource:0}: Error finding container fabaa23439d4e8af5109663a402eb11dc5c8fcefaa3e52f0d2d7db29038e1cb5: Status 404 returned error can't find the container with id fabaa23439d4e8af5109663a402eb11dc5c8fcefaa3e52f0d2d7db29038e1cb5 Jul 30 00:49:42.323623 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:42.323586 2564 generic.go:356] "Generic (PLEG): container finished" podID="a335e848-73cc-4844-8007-2bb942c21949" containerID="c6a01e65eac62d0f94746e3d4a4a7e1a7f2636818e75662aeeb1df60d4591322" exitCode=0 Jul 30 00:49:42.324044 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:42.323654 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" event={"ID":"a335e848-73cc-4844-8007-2bb942c21949","Type":"ContainerDied","Data":"c6a01e65eac62d0f94746e3d4a4a7e1a7f2636818e75662aeeb1df60d4591322"} Jul 30 00:49:42.324044 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:42.323694 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" event={"ID":"a335e848-73cc-4844-8007-2bb942c21949","Type":"ContainerStarted","Data":"fabaa23439d4e8af5109663a402eb11dc5c8fcefaa3e52f0d2d7db29038e1cb5"} Jul 30 00:49:43.329806 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:43.329770 2564 generic.go:356] "Generic (PLEG): container finished" podID="a335e848-73cc-4844-8007-2bb942c21949" containerID="4d04853f490addb2e88597f0241d1a72d106298e560a00f92967b700cfb1a6dc" exitCode=0 Jul 30 00:49:43.330257 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:43.329853 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" event={"ID":"a335e848-73cc-4844-8007-2bb942c21949","Type":"ContainerDied","Data":"4d04853f490addb2e88597f0241d1a72d106298e560a00f92967b700cfb1a6dc"} Jul 30 00:49:44.338439 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:44.338410 2564 generic.go:356] "Generic (PLEG): container finished" podID="a335e848-73cc-4844-8007-2bb942c21949" containerID="2938f402c4d0772332334d211790838807844a6263944887037f68164688befd" exitCode=0 Jul 30 00:49:44.338832 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:44.338498 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" event={"ID":"a335e848-73cc-4844-8007-2bb942c21949","Type":"ContainerDied","Data":"2938f402c4d0772332334d211790838807844a6263944887037f68164688befd"} Jul 30 00:49:45.470199 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:45.470172 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" Jul 30 00:49:45.578122 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:45.578080 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a335e848-73cc-4844-8007-2bb942c21949-bundle\") pod \"a335e848-73cc-4844-8007-2bb942c21949\" (UID: \"a335e848-73cc-4844-8007-2bb942c21949\") " Jul 30 00:49:45.578122 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:45.578126 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a335e848-73cc-4844-8007-2bb942c21949-util\") pod \"a335e848-73cc-4844-8007-2bb942c21949\" (UID: \"a335e848-73cc-4844-8007-2bb942c21949\") " Jul 30 00:49:45.578338 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:45.578163 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-js7n9\" (UniqueName: \"kubernetes.io/projected/a335e848-73cc-4844-8007-2bb942c21949-kube-api-access-js7n9\") pod \"a335e848-73cc-4844-8007-2bb942c21949\" (UID: \"a335e848-73cc-4844-8007-2bb942c21949\") " Jul 30 00:49:45.578829 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:45.578799 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a335e848-73cc-4844-8007-2bb942c21949-bundle" (OuterVolumeSpecName: "bundle") pod "a335e848-73cc-4844-8007-2bb942c21949" (UID: "a335e848-73cc-4844-8007-2bb942c21949"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:49:45.580263 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:45.580237 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a335e848-73cc-4844-8007-2bb942c21949-kube-api-access-js7n9" (OuterVolumeSpecName: "kube-api-access-js7n9") pod "a335e848-73cc-4844-8007-2bb942c21949" (UID: "a335e848-73cc-4844-8007-2bb942c21949"). InnerVolumeSpecName "kube-api-access-js7n9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:49:45.584141 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:45.584119 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a335e848-73cc-4844-8007-2bb942c21949-util" (OuterVolumeSpecName: "util") pod "a335e848-73cc-4844-8007-2bb942c21949" (UID: "a335e848-73cc-4844-8007-2bb942c21949"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:49:45.679778 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:45.679750 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a335e848-73cc-4844-8007-2bb942c21949-bundle\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:49:45.679778 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:45.679776 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a335e848-73cc-4844-8007-2bb942c21949-util\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:49:45.679972 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:45.679789 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-js7n9\" (UniqueName: \"kubernetes.io/projected/a335e848-73cc-4844-8007-2bb942c21949-kube-api-access-js7n9\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:49:46.349357 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:46.349326 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" Jul 30 00:49:46.349357 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:46.349352 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpgcxd" event={"ID":"a335e848-73cc-4844-8007-2bb942c21949","Type":"ContainerDied","Data":"fabaa23439d4e8af5109663a402eb11dc5c8fcefaa3e52f0d2d7db29038e1cb5"} Jul 30 00:49:46.349568 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:46.349400 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fabaa23439d4e8af5109663a402eb11dc5c8fcefaa3e52f0d2d7db29038e1cb5" Jul 30 00:49:53.088965 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:53.088925 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:53.189664 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:53.189630 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:54.062671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.062635 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw"] Jul 30 00:49:54.062985 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.062970 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a335e848-73cc-4844-8007-2bb942c21949" containerName="extract" Jul 30 00:49:54.062985 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.062986 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a335e848-73cc-4844-8007-2bb942c21949" containerName="extract" Jul 30 00:49:54.063084 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.063003 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a335e848-73cc-4844-8007-2bb942c21949" containerName="util" Jul 30 00:49:54.063084 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.063010 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a335e848-73cc-4844-8007-2bb942c21949" containerName="util" Jul 30 00:49:54.063084 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.063015 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a335e848-73cc-4844-8007-2bb942c21949" containerName="pull" Jul 30 00:49:54.063084 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.063021 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a335e848-73cc-4844-8007-2bb942c21949" containerName="pull" Jul 30 00:49:54.063084 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.063079 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a335e848-73cc-4844-8007-2bb942c21949" containerName="extract" Jul 30 00:49:54.066481 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.066452 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.069556 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.069281 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-585w4\"" Jul 30 00:49:54.069556 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.069302 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 30 00:49:54.069556 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.069512 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 30 00:49:54.069781 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.069559 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 30 00:49:54.069781 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.069566 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 30 00:49:54.089717 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.089678 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw"] Jul 30 00:49:54.153037 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.153001 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b2dfe990-5238-4ad7-8c27-43dfde9eda24-webhook-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-htpnw\" (UID: \"b2dfe990-5238-4ad7-8c27-43dfde9eda24\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.153037 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.153048 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxpcj\" (UniqueName: \"kubernetes.io/projected/b2dfe990-5238-4ad7-8c27-43dfde9eda24-kube-api-access-qxpcj\") pod \"opendatahub-operator-controller-manager-746b4b6684-htpnw\" (UID: \"b2dfe990-5238-4ad7-8c27-43dfde9eda24\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.153261 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.153165 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b2dfe990-5238-4ad7-8c27-43dfde9eda24-apiservice-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-htpnw\" (UID: \"b2dfe990-5238-4ad7-8c27-43dfde9eda24\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.153261 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.153201 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b2dfe990-5238-4ad7-8c27-43dfde9eda24-manifests\") pod \"opendatahub-operator-controller-manager-746b4b6684-htpnw\" (UID: \"b2dfe990-5238-4ad7-8c27-43dfde9eda24\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.153261 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.153218 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2dfe990-5238-4ad7-8c27-43dfde9eda24-tmp\") pod \"opendatahub-operator-controller-manager-746b4b6684-htpnw\" (UID: \"b2dfe990-5238-4ad7-8c27-43dfde9eda24\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.254003 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.253943 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b2dfe990-5238-4ad7-8c27-43dfde9eda24-webhook-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-htpnw\" (UID: \"b2dfe990-5238-4ad7-8c27-43dfde9eda24\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.254003 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.253997 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxpcj\" (UniqueName: \"kubernetes.io/projected/b2dfe990-5238-4ad7-8c27-43dfde9eda24-kube-api-access-qxpcj\") pod \"opendatahub-operator-controller-manager-746b4b6684-htpnw\" (UID: \"b2dfe990-5238-4ad7-8c27-43dfde9eda24\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.254320 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.254051 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b2dfe990-5238-4ad7-8c27-43dfde9eda24-apiservice-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-htpnw\" (UID: \"b2dfe990-5238-4ad7-8c27-43dfde9eda24\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.254320 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.254076 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b2dfe990-5238-4ad7-8c27-43dfde9eda24-manifests\") pod \"opendatahub-operator-controller-manager-746b4b6684-htpnw\" (UID: \"b2dfe990-5238-4ad7-8c27-43dfde9eda24\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.254320 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.254091 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2dfe990-5238-4ad7-8c27-43dfde9eda24-tmp\") pod \"opendatahub-operator-controller-manager-746b4b6684-htpnw\" (UID: \"b2dfe990-5238-4ad7-8c27-43dfde9eda24\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.254540 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.254512 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b2dfe990-5238-4ad7-8c27-43dfde9eda24-manifests\") pod \"opendatahub-operator-controller-manager-746b4b6684-htpnw\" (UID: \"b2dfe990-5238-4ad7-8c27-43dfde9eda24\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.256415 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.256390 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2dfe990-5238-4ad7-8c27-43dfde9eda24-tmp\") pod \"opendatahub-operator-controller-manager-746b4b6684-htpnw\" (UID: \"b2dfe990-5238-4ad7-8c27-43dfde9eda24\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.256644 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.256626 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b2dfe990-5238-4ad7-8c27-43dfde9eda24-apiservice-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-htpnw\" (UID: \"b2dfe990-5238-4ad7-8c27-43dfde9eda24\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.256644 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.256635 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b2dfe990-5238-4ad7-8c27-43dfde9eda24-webhook-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-htpnw\" (UID: \"b2dfe990-5238-4ad7-8c27-43dfde9eda24\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.262431 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.262407 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxpcj\" (UniqueName: \"kubernetes.io/projected/b2dfe990-5238-4ad7-8c27-43dfde9eda24-kube-api-access-qxpcj\") pod \"opendatahub-operator-controller-manager-746b4b6684-htpnw\" (UID: \"b2dfe990-5238-4ad7-8c27-43dfde9eda24\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.377470 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.377434 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:49:54.517471 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.517433 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw"] Jul 30 00:49:54.521437 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:49:54.521405 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2dfe990_5238_4ad7_8c27_43dfde9eda24.slice/crio-82ffc27f1a0b0c5a8735ab2c83e6de1dc24323e5f7eda5def10f51f587f4de08 WatchSource:0}: Error finding container 82ffc27f1a0b0c5a8735ab2c83e6de1dc24323e5f7eda5def10f51f587f4de08: Status 404 returned error can't find the container with id 82ffc27f1a0b0c5a8735ab2c83e6de1dc24323e5f7eda5def10f51f587f4de08 Jul 30 00:49:54.634816 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.634719 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:54.735244 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.735212 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:54.836680 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.836646 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:54.937695 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:54.937605 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:55.393273 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:55.393236 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" event={"ID":"b2dfe990-5238-4ad7-8c27-43dfde9eda24","Type":"ContainerStarted","Data":"82ffc27f1a0b0c5a8735ab2c83e6de1dc24323e5f7eda5def10f51f587f4de08"} Jul 30 00:49:58.412395 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:58.412360 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" event={"ID":"b2dfe990-5238-4ad7-8c27-43dfde9eda24","Type":"ContainerStarted","Data":"f4fad65cc997833147e10d2188333fde7f82be10d114f7a22363106339910d17"} Jul 30 00:49:59.419407 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:59.419372 2564 generic.go:356] "Generic (PLEG): container finished" podID="b2dfe990-5238-4ad7-8c27-43dfde9eda24" containerID="f4fad65cc997833147e10d2188333fde7f82be10d114f7a22363106339910d17" exitCode=0 Jul 30 00:49:59.419755 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:49:59.419452 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" event={"ID":"b2dfe990-5238-4ad7-8c27-43dfde9eda24","Type":"ContainerDied","Data":"f4fad65cc997833147e10d2188333fde7f82be10d114f7a22363106339910d17"} Jul 30 00:50:00.425647 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:00.425605 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" event={"ID":"b2dfe990-5238-4ad7-8c27-43dfde9eda24","Type":"ContainerStarted","Data":"360aaac3365b0ed86b0d74a8c727b7c590401e3055b92173c76eefbb5241ce93"} Jul 30 00:50:00.426049 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:00.425704 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:50:00.446919 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:00.446841 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" podStartSLOduration=1.313749375 podStartE2EDuration="6.446820299s" totalImagesPullingTime="4.045586761s" totalInitContainerRuntime="1.087484163s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:49:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:50:00.445302846 +0000 UTC m=+403.246259991" watchObservedRunningTime="2026-07-30 00:50:00.446820299 +0000 UTC m=+403.247777411" Jul 30 00:50:11.202701 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.202662 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x"] Jul 30 00:50:11.206093 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.206075 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:11.209342 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.209321 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 30 00:50:11.209593 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.209566 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 30 00:50:11.209687 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.209572 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 30 00:50:11.209687 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.209607 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2rjqc\"" Jul 30 00:50:11.209687 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.209636 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 30 00:50:11.209856 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.209696 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 30 00:50:11.213832 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.213801 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x"] Jul 30 00:50:11.308524 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.308485 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8c7cf3b-633e-4d2e-9586-e4995d2cc743-cert\") pod \"lws-controller-manager-6d674c9db-9qm8x\" (UID: \"f8c7cf3b-633e-4d2e-9586-e4995d2cc743\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:11.308524 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.308527 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8c7cf3b-633e-4d2e-9586-e4995d2cc743-metrics-cert\") pod \"lws-controller-manager-6d674c9db-9qm8x\" (UID: \"f8c7cf3b-633e-4d2e-9586-e4995d2cc743\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:11.308741 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.308594 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhm9s\" (UniqueName: \"kubernetes.io/projected/f8c7cf3b-633e-4d2e-9586-e4995d2cc743-kube-api-access-xhm9s\") pod \"lws-controller-manager-6d674c9db-9qm8x\" (UID: \"f8c7cf3b-633e-4d2e-9586-e4995d2cc743\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:11.308741 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.308690 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f8c7cf3b-633e-4d2e-9586-e4995d2cc743-manager-config\") pod \"lws-controller-manager-6d674c9db-9qm8x\" (UID: \"f8c7cf3b-633e-4d2e-9586-e4995d2cc743\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:11.409412 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.409364 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8c7cf3b-633e-4d2e-9586-e4995d2cc743-metrics-cert\") pod \"lws-controller-manager-6d674c9db-9qm8x\" (UID: \"f8c7cf3b-633e-4d2e-9586-e4995d2cc743\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:11.409543 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.409427 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhm9s\" (UniqueName: \"kubernetes.io/projected/f8c7cf3b-633e-4d2e-9586-e4995d2cc743-kube-api-access-xhm9s\") pod \"lws-controller-manager-6d674c9db-9qm8x\" (UID: \"f8c7cf3b-633e-4d2e-9586-e4995d2cc743\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:11.409543 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.409484 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f8c7cf3b-633e-4d2e-9586-e4995d2cc743-manager-config\") pod \"lws-controller-manager-6d674c9db-9qm8x\" (UID: \"f8c7cf3b-633e-4d2e-9586-e4995d2cc743\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:11.409543 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.409526 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8c7cf3b-633e-4d2e-9586-e4995d2cc743-cert\") pod \"lws-controller-manager-6d674c9db-9qm8x\" (UID: \"f8c7cf3b-633e-4d2e-9586-e4995d2cc743\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:11.410285 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.410252 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f8c7cf3b-633e-4d2e-9586-e4995d2cc743-manager-config\") pod \"lws-controller-manager-6d674c9db-9qm8x\" (UID: \"f8c7cf3b-633e-4d2e-9586-e4995d2cc743\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:11.412010 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.411977 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8c7cf3b-633e-4d2e-9586-e4995d2cc743-cert\") pod \"lws-controller-manager-6d674c9db-9qm8x\" (UID: \"f8c7cf3b-633e-4d2e-9586-e4995d2cc743\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:11.412142 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.412020 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8c7cf3b-633e-4d2e-9586-e4995d2cc743-metrics-cert\") pod \"lws-controller-manager-6d674c9db-9qm8x\" (UID: \"f8c7cf3b-633e-4d2e-9586-e4995d2cc743\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:11.417224 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.417198 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhm9s\" (UniqueName: \"kubernetes.io/projected/f8c7cf3b-633e-4d2e-9586-e4995d2cc743-kube-api-access-xhm9s\") pod \"lws-controller-manager-6d674c9db-9qm8x\" (UID: \"f8c7cf3b-633e-4d2e-9586-e4995d2cc743\") " pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:11.432845 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.432818 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-htpnw" Jul 30 00:50:11.516560 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.516479 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:11.652541 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:11.652517 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x"] Jul 30 00:50:11.657770 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:50:11.657693 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c7cf3b_633e_4d2e_9586_e4995d2cc743.slice/crio-628e70ac0831b85d66665d715970f86bb61e3584f08dbd29507b193e89fa6774 WatchSource:0}: Error finding container 628e70ac0831b85d66665d715970f86bb61e3584f08dbd29507b193e89fa6774: Status 404 returned error can't find the container with id 628e70ac0831b85d66665d715970f86bb61e3584f08dbd29507b193e89fa6774 Jul 30 00:50:12.482455 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:12.482425 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" event={"ID":"f8c7cf3b-633e-4d2e-9586-e4995d2cc743","Type":"ContainerStarted","Data":"628e70ac0831b85d66665d715970f86bb61e3584f08dbd29507b193e89fa6774"} Jul 30 00:50:13.203875 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:13.203831 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:13.360507 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:13.360463 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:13.507695 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:13.507603 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:13.726124 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:13.726082 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:13.862308 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:13.862273 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:14.181963 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:14.181866 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:14.253261 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:14.253227 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:14.325411 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:14.325377 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:14.493651 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:14.493562 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" event={"ID":"f8c7cf3b-633e-4d2e-9586-e4995d2cc743","Type":"ContainerStarted","Data":"c1970ff773e5b60bc4c5bbfbbb0f7b9ba03fc51da70954cb6590a6630ee7767c"} Jul 30 00:50:14.493796 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:14.493698 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:14.509356 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:14.509321 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:14.518390 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:14.518330 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" podStartSLOduration=1.619146186 podStartE2EDuration="3.518313411s" totalImagesPullingTime="1.899167225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:50:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:50:14.515750894 +0000 UTC m=+417.316707998" watchObservedRunningTime="2026-07-30 00:50:14.518313411 +0000 UTC m=+417.319270517" Jul 30 00:50:15.045127 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:15.045085 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:15.247013 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:15.246980 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:15.346399 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:15.346362 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:16.953402 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:16.953362 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:17.017979 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:17.017944 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:21.799004 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:21.798964 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:21.905388 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:21.905351 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:22.101513 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.101472 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76"] Jul 30 00:50:22.105655 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.105628 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.108253 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.108217 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 30 00:50:22.108864 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.108831 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-vgtbw\"" Jul 30 00:50:22.121415 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.121382 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76"] Jul 30 00:50:22.145594 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.145561 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:22.210530 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.210488 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575ac028-f779-46c9-a84e-2476fee11fe8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.210735 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.210539 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575ac028-f779-46c9-a84e-2476fee11fe8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.210735 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.210566 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575ac028-f779-46c9-a84e-2476fee11fe8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.210735 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.210596 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575ac028-f779-46c9-a84e-2476fee11fe8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.210735 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.210626 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575ac028-f779-46c9-a84e-2476fee11fe8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.210735 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.210646 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575ac028-f779-46c9-a84e-2476fee11fe8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.210735 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.210673 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9tqs\" (UniqueName: \"kubernetes.io/projected/575ac028-f779-46c9-a84e-2476fee11fe8-kube-api-access-m9tqs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.211081 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.210750 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575ac028-f779-46c9-a84e-2476fee11fe8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.211081 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.210865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575ac028-f779-46c9-a84e-2476fee11fe8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.302545 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.302509 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j"] Jul 30 00:50:22.305207 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.305185 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.307576 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.307549 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 30 00:50:22.307723 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.307584 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 30 00:50:22.307723 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.307682 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cd49j\"" Jul 30 00:50:22.307918 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.307885 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 30 00:50:22.311513 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.311483 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575ac028-f779-46c9-a84e-2476fee11fe8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.311671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.311531 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9tqs\" (UniqueName: \"kubernetes.io/projected/575ac028-f779-46c9-a84e-2476fee11fe8-kube-api-access-m9tqs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.311671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.311562 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575ac028-f779-46c9-a84e-2476fee11fe8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.311671 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.311611 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575ac028-f779-46c9-a84e-2476fee11fe8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.311844 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.311700 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575ac028-f779-46c9-a84e-2476fee11fe8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.311844 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.311731 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575ac028-f779-46c9-a84e-2476fee11fe8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.311844 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.311755 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575ac028-f779-46c9-a84e-2476fee11fe8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.311844 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.311784 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575ac028-f779-46c9-a84e-2476fee11fe8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.311844 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.311833 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575ac028-f779-46c9-a84e-2476fee11fe8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.312504 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.312230 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575ac028-f779-46c9-a84e-2476fee11fe8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.312504 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.312305 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575ac028-f779-46c9-a84e-2476fee11fe8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.312504 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.312368 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575ac028-f779-46c9-a84e-2476fee11fe8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.312504 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.312468 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575ac028-f779-46c9-a84e-2476fee11fe8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.312858 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.312662 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575ac028-f779-46c9-a84e-2476fee11fe8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.314936 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.314889 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575ac028-f779-46c9-a84e-2476fee11fe8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.315049 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.314913 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575ac028-f779-46c9-a84e-2476fee11fe8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.325247 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.324803 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j"] Jul 30 00:50:22.332477 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.327193 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575ac028-f779-46c9-a84e-2476fee11fe8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.333306 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.333275 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9tqs\" (UniqueName: \"kubernetes.io/projected/575ac028-f779-46c9-a84e-2476fee11fe8-kube-api-access-m9tqs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76\" (UID: \"575ac028-f779-46c9-a84e-2476fee11fe8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.343824 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.343787 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:22.413179 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.413082 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/330c2ba1-c814-4f05-85c9-de6508091b8c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.413179 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.413128 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/330c2ba1-c814-4f05-85c9-de6508091b8c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.413179 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.413155 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46njz\" (UniqueName: \"kubernetes.io/projected/330c2ba1-c814-4f05-85c9-de6508091b8c-kube-api-access-46njz\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.413463 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.413206 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/330c2ba1-c814-4f05-85c9-de6508091b8c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.413463 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.413229 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/330c2ba1-c814-4f05-85c9-de6508091b8c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.413463 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.413348 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/330c2ba1-c814-4f05-85c9-de6508091b8c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.413463 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.413398 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/330c2ba1-c814-4f05-85c9-de6508091b8c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.420062 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.420029 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:22.514511 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.514472 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/330c2ba1-c814-4f05-85c9-de6508091b8c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.514511 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.514520 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/330c2ba1-c814-4f05-85c9-de6508091b8c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.514837 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.514811 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46njz\" (UniqueName: \"kubernetes.io/projected/330c2ba1-c814-4f05-85c9-de6508091b8c-kube-api-access-46njz\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.514913 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.514884 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/330c2ba1-c814-4f05-85c9-de6508091b8c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.514973 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.514936 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/330c2ba1-c814-4f05-85c9-de6508091b8c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.515392 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.515038 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/330c2ba1-c814-4f05-85c9-de6508091b8c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.515392 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.515074 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/330c2ba1-c814-4f05-85c9-de6508091b8c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.516097 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.516076 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/330c2ba1-c814-4f05-85c9-de6508091b8c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.517443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.517397 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/330c2ba1-c814-4f05-85c9-de6508091b8c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.517443 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.517405 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/330c2ba1-c814-4f05-85c9-de6508091b8c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.518927 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.518328 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/330c2ba1-c814-4f05-85c9-de6508091b8c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.518927 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.518573 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/330c2ba1-c814-4f05-85c9-de6508091b8c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.528713 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.528662 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/330c2ba1-c814-4f05-85c9-de6508091b8c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.529196 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.529173 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46njz\" (UniqueName: \"kubernetes.io/projected/330c2ba1-c814-4f05-85c9-de6508091b8c-kube-api-access-46njz\") pod \"istiod-openshift-gateway-5677ccb49b-g2m8j\" (UID: \"330c2ba1-c814-4f05-85c9-de6508091b8c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.566866 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.566830 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76"] Jul 30 00:50:22.569601 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:50:22.569571 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod575ac028_f779_46c9_a84e_2476fee11fe8.slice/crio-ff4834c90476ba3c3221204bb6240386c15ed7f4f8e1e0b58a69b13ce73f261b WatchSource:0}: Error finding container ff4834c90476ba3c3221204bb6240386c15ed7f4f8e1e0b58a69b13ce73f261b: Status 404 returned error can't find the container with id ff4834c90476ba3c3221204bb6240386c15ed7f4f8e1e0b58a69b13ce73f261b Jul 30 00:50:22.604106 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.604065 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:22.615990 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.615954 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:22.759062 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:22.759035 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j"] Jul 30 00:50:22.761286 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:50:22.761258 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod330c2ba1_c814_4f05_85c9_de6508091b8c.slice/crio-93b8bc2aaa6d6b87266946fe90d8e05aa6e4111ea437d8440fa05f8b840859db WatchSource:0}: Error finding container 93b8bc2aaa6d6b87266946fe90d8e05aa6e4111ea437d8440fa05f8b840859db: Status 404 returned error can't find the container with id 93b8bc2aaa6d6b87266946fe90d8e05aa6e4111ea437d8440fa05f8b840859db Jul 30 00:50:23.545131 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:23.545091 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" event={"ID":"330c2ba1-c814-4f05-85c9-de6508091b8c","Type":"ContainerStarted","Data":"93b8bc2aaa6d6b87266946fe90d8e05aa6e4111ea437d8440fa05f8b840859db"} Jul 30 00:50:23.548454 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:23.548412 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" event={"ID":"575ac028-f779-46c9-a84e-2476fee11fe8","Type":"ContainerStarted","Data":"ff4834c90476ba3c3221204bb6240386c15ed7f4f8e1e0b58a69b13ce73f261b"} Jul 30 00:50:24.872727 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:24.872646 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:24.954519 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:24.954474 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:25.031110 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:25.031019 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:25.444470 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:25.444426 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:25.500808 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:25.500779 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6d674c9db-9qm8x" Jul 30 00:50:26.130749 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:26.130704 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:26.274886 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:26.274849 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:26.313350 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:26.313312 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 00:50:26.313502 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:26.313390 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 00:50:26.318413 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:26.318339 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 00:50:26.318572 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:26.318430 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 00:50:26.318572 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:26.318474 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 00:50:26.365678 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:26.365639 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:26.568113 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:26.568075 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" event={"ID":"575ac028-f779-46c9-a84e-2476fee11fe8","Type":"ContainerStarted","Data":"838dc9b50ed90471fd29818e36a53a46e56d4a03290fb7671a7bf4720cc2e896"} Jul 30 00:50:26.570239 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:26.570201 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" event={"ID":"330c2ba1-c814-4f05-85c9-de6508091b8c","Type":"ContainerStarted","Data":"ad796dd6ffb77a64f644ad6e26307bfe7ca61a3619f95488c51ebd2361115324"} Jul 30 00:50:26.570421 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:26.570329 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:50:26.592200 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:26.592137 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" podStartSLOduration=0.845516222 podStartE2EDuration="4.592114519s" totalImagesPullingTime="3.746598297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:50:26.590934765 +0000 UTC m=+429.391891871" watchObservedRunningTime="2026-07-30 00:50:26.592114519 +0000 UTC m=+429.393071630" Jul 30 00:50:26.614267 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:26.614195 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" podStartSLOduration=1.064506737 podStartE2EDuration="4.614171049s" totalImagesPullingTime="3.549664312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:50:26.612133239 +0000 UTC m=+429.413090347" watchObservedRunningTime="2026-07-30 00:50:26.614171049 +0000 UTC m=+429.415128156" Jul 30 00:50:27.421070 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:27.421026 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:27.425815 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:27.425782 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:27.575371 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:27.575335 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:27.576413 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:27.576390 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76" Jul 30 00:50:27.577225 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:50:27.577203 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g2m8j" Jul 30 00:51:14.588702 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:14.588658 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:14.651406 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:14.651369 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:14.748677 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:14.748642 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:17.568835 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:17.568800 2564 scope.go:117] "RemoveContainer" containerID="1cf4713e701f4b598be2aac98f4fb48ce90c693d890326e127b05c7e48e8f90c" Jul 30 00:51:34.090629 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:34.090593 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:34.192156 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:34.192120 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:34.291732 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:34.291698 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:35.464495 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:35.464461 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:35.514485 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:35.514452 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:38.522482 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.522443 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p"] Jul 30 00:51:38.530528 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.530495 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" Jul 30 00:51:38.532936 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.532888 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 30 00:51:38.533107 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.532888 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 30 00:51:38.533799 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.533781 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5t6b4\"" Jul 30 00:51:38.534004 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.533981 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p"] Jul 30 00:51:38.592857 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.592817 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07942485-9a87-436d-8ab9-72797e8e36ae-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p\" (UID: \"07942485-9a87-436d-8ab9-72797e8e36ae\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" Jul 30 00:51:38.592857 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.592860 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtgtx\" (UniqueName: \"kubernetes.io/projected/07942485-9a87-436d-8ab9-72797e8e36ae-kube-api-access-dtgtx\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p\" (UID: \"07942485-9a87-436d-8ab9-72797e8e36ae\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" Jul 30 00:51:38.593111 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.592992 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07942485-9a87-436d-8ab9-72797e8e36ae-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p\" (UID: \"07942485-9a87-436d-8ab9-72797e8e36ae\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" Jul 30 00:51:38.694073 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.694028 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07942485-9a87-436d-8ab9-72797e8e36ae-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p\" (UID: \"07942485-9a87-436d-8ab9-72797e8e36ae\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" Jul 30 00:51:38.694249 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.694120 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07942485-9a87-436d-8ab9-72797e8e36ae-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p\" (UID: \"07942485-9a87-436d-8ab9-72797e8e36ae\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" Jul 30 00:51:38.694249 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.694139 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtgtx\" (UniqueName: \"kubernetes.io/projected/07942485-9a87-436d-8ab9-72797e8e36ae-kube-api-access-dtgtx\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p\" (UID: \"07942485-9a87-436d-8ab9-72797e8e36ae\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" Jul 30 00:51:38.694423 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.694406 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07942485-9a87-436d-8ab9-72797e8e36ae-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p\" (UID: \"07942485-9a87-436d-8ab9-72797e8e36ae\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" Jul 30 00:51:38.694515 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.694493 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07942485-9a87-436d-8ab9-72797e8e36ae-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p\" (UID: \"07942485-9a87-436d-8ab9-72797e8e36ae\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" Jul 30 00:51:38.704241 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.704203 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtgtx\" (UniqueName: \"kubernetes.io/projected/07942485-9a87-436d-8ab9-72797e8e36ae-kube-api-access-dtgtx\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p\" (UID: \"07942485-9a87-436d-8ab9-72797e8e36ae\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" Jul 30 00:51:38.841768 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.841734 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" Jul 30 00:51:38.978370 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:38.978349 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p"] Jul 30 00:51:38.980436 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:51:38.980409 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07942485_9a87_436d_8ab9_72797e8e36ae.slice/crio-a2c1d762b39459f051c19735c523da5ea442da1f1ba6169db598e751621ca4bf WatchSource:0}: Error finding container a2c1d762b39459f051c19735c523da5ea442da1f1ba6169db598e751621ca4bf: Status 404 returned error can't find the container with id a2c1d762b39459f051c19735c523da5ea442da1f1ba6169db598e751621ca4bf Jul 30 00:51:39.117263 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.117171 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks"] Jul 30 00:51:39.120764 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.120746 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" Jul 30 00:51:39.129254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.129223 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks"] Jul 30 00:51:39.199701 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.199665 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks\" (UID: \"2c4ece51-398a-41c3-9cd8-87d4cce68a8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" Jul 30 00:51:39.199701 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.199708 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks\" (UID: \"2c4ece51-398a-41c3-9cd8-87d4cce68a8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" Jul 30 00:51:39.200020 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.199734 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ph74\" (UniqueName: \"kubernetes.io/projected/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-kube-api-access-4ph74\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks\" (UID: \"2c4ece51-398a-41c3-9cd8-87d4cce68a8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" Jul 30 00:51:39.301089 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.301046 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ph74\" (UniqueName: \"kubernetes.io/projected/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-kube-api-access-4ph74\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks\" (UID: \"2c4ece51-398a-41c3-9cd8-87d4cce68a8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" Jul 30 00:51:39.301268 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.301160 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks\" (UID: \"2c4ece51-398a-41c3-9cd8-87d4cce68a8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" Jul 30 00:51:39.301268 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.301194 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks\" (UID: \"2c4ece51-398a-41c3-9cd8-87d4cce68a8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" Jul 30 00:51:39.301562 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.301546 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks\" (UID: \"2c4ece51-398a-41c3-9cd8-87d4cce68a8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" Jul 30 00:51:39.301613 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.301578 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks\" (UID: \"2c4ece51-398a-41c3-9cd8-87d4cce68a8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" Jul 30 00:51:39.309947 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.309921 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ph74\" (UniqueName: \"kubernetes.io/projected/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-kube-api-access-4ph74\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks\" (UID: \"2c4ece51-398a-41c3-9cd8-87d4cce68a8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" Jul 30 00:51:39.469083 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.468983 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" Jul 30 00:51:39.614516 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.614490 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks"] Jul 30 00:51:39.616592 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:51:39.616557 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c4ece51_398a_41c3_9cd8_87d4cce68a8b.slice/crio-de5e3c93b6a3b9f499b83f9f60481cb2ea86ef8ba956f055b5b9b8efc0723b5e WatchSource:0}: Error finding container de5e3c93b6a3b9f499b83f9f60481cb2ea86ef8ba956f055b5b9b8efc0723b5e: Status 404 returned error can't find the container with id de5e3c93b6a3b9f499b83f9f60481cb2ea86ef8ba956f055b5b9b8efc0723b5e Jul 30 00:51:39.725283 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.725249 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc"] Jul 30 00:51:39.728795 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.728774 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" Jul 30 00:51:39.736330 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.736298 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc"] Jul 30 00:51:39.805297 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.805259 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzf9m\" (UniqueName: \"kubernetes.io/projected/721d5439-21ee-46a5-97a8-b34345828dd0-kube-api-access-mzf9m\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc\" (UID: \"721d5439-21ee-46a5-97a8-b34345828dd0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" Jul 30 00:51:39.805468 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.805333 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/721d5439-21ee-46a5-97a8-b34345828dd0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc\" (UID: \"721d5439-21ee-46a5-97a8-b34345828dd0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" Jul 30 00:51:39.805468 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.805444 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/721d5439-21ee-46a5-97a8-b34345828dd0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc\" (UID: \"721d5439-21ee-46a5-97a8-b34345828dd0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" Jul 30 00:51:39.906244 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.906209 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/721d5439-21ee-46a5-97a8-b34345828dd0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc\" (UID: \"721d5439-21ee-46a5-97a8-b34345828dd0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" Jul 30 00:51:39.906244 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.906250 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzf9m\" (UniqueName: \"kubernetes.io/projected/721d5439-21ee-46a5-97a8-b34345828dd0-kube-api-access-mzf9m\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc\" (UID: \"721d5439-21ee-46a5-97a8-b34345828dd0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" Jul 30 00:51:39.906465 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.906291 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/721d5439-21ee-46a5-97a8-b34345828dd0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc\" (UID: \"721d5439-21ee-46a5-97a8-b34345828dd0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" Jul 30 00:51:39.906672 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.906653 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/721d5439-21ee-46a5-97a8-b34345828dd0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc\" (UID: \"721d5439-21ee-46a5-97a8-b34345828dd0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" Jul 30 00:51:39.906712 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.906661 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/721d5439-21ee-46a5-97a8-b34345828dd0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc\" (UID: \"721d5439-21ee-46a5-97a8-b34345828dd0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" Jul 30 00:51:39.914463 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.914434 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzf9m\" (UniqueName: \"kubernetes.io/projected/721d5439-21ee-46a5-97a8-b34345828dd0-kube-api-access-mzf9m\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc\" (UID: \"721d5439-21ee-46a5-97a8-b34345828dd0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" Jul 30 00:51:39.943389 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.943359 2564 generic.go:356] "Generic (PLEG): container finished" podID="07942485-9a87-436d-8ab9-72797e8e36ae" containerID="d19a5f6827a01200d3743d3ea33383c9830ae1217c307abe6f56947a56f9e164" exitCode=0 Jul 30 00:51:39.943576 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.943448 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" event={"ID":"07942485-9a87-436d-8ab9-72797e8e36ae","Type":"ContainerDied","Data":"d19a5f6827a01200d3743d3ea33383c9830ae1217c307abe6f56947a56f9e164"} Jul 30 00:51:39.943576 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.943513 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" event={"ID":"07942485-9a87-436d-8ab9-72797e8e36ae","Type":"ContainerStarted","Data":"a2c1d762b39459f051c19735c523da5ea442da1f1ba6169db598e751621ca4bf"} Jul 30 00:51:39.945150 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.945117 2564 generic.go:356] "Generic (PLEG): container finished" podID="2c4ece51-398a-41c3-9cd8-87d4cce68a8b" containerID="b52c33645db88bb48b0b5cc283947ea80d4ef6b7562ff5e0aeaeb264a1c26762" exitCode=0 Jul 30 00:51:39.945261 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.945151 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" event={"ID":"2c4ece51-398a-41c3-9cd8-87d4cce68a8b","Type":"ContainerDied","Data":"b52c33645db88bb48b0b5cc283947ea80d4ef6b7562ff5e0aeaeb264a1c26762"} Jul 30 00:51:39.945261 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:39.945200 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" event={"ID":"2c4ece51-398a-41c3-9cd8-87d4cce68a8b","Type":"ContainerStarted","Data":"de5e3c93b6a3b9f499b83f9f60481cb2ea86ef8ba956f055b5b9b8efc0723b5e"} Jul 30 00:51:40.092281 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.092244 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" Jul 30 00:51:40.238752 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:51:40.238708 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod721d5439_21ee_46a5_97a8_b34345828dd0.slice/crio-750ed8d1497e2a62ad49cd82c5fa185045be6428c5657430aa9f762bbee97d0b WatchSource:0}: Error finding container 750ed8d1497e2a62ad49cd82c5fa185045be6428c5657430aa9f762bbee97d0b: Status 404 returned error can't find the container with id 750ed8d1497e2a62ad49cd82c5fa185045be6428c5657430aa9f762bbee97d0b Jul 30 00:51:40.239120 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.239085 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc"] Jul 30 00:51:40.325748 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.325701 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb"] Jul 30 00:51:40.329465 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.329436 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" Jul 30 00:51:40.337916 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.337860 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb"] Jul 30 00:51:40.410648 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.410549 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1503a956-5e65-4c0b-8d68-d932e789d4a6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb\" (UID: \"1503a956-5e65-4c0b-8d68-d932e789d4a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" Jul 30 00:51:40.410648 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.410604 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1503a956-5e65-4c0b-8d68-d932e789d4a6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb\" (UID: \"1503a956-5e65-4c0b-8d68-d932e789d4a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" Jul 30 00:51:40.410842 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.410683 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td2p7\" (UniqueName: \"kubernetes.io/projected/1503a956-5e65-4c0b-8d68-d932e789d4a6-kube-api-access-td2p7\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb\" (UID: \"1503a956-5e65-4c0b-8d68-d932e789d4a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" Jul 30 00:51:40.511367 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.511325 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1503a956-5e65-4c0b-8d68-d932e789d4a6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb\" (UID: \"1503a956-5e65-4c0b-8d68-d932e789d4a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" Jul 30 00:51:40.511553 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.511388 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1503a956-5e65-4c0b-8d68-d932e789d4a6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb\" (UID: \"1503a956-5e65-4c0b-8d68-d932e789d4a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" Jul 30 00:51:40.511553 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.511439 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td2p7\" (UniqueName: \"kubernetes.io/projected/1503a956-5e65-4c0b-8d68-d932e789d4a6-kube-api-access-td2p7\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb\" (UID: \"1503a956-5e65-4c0b-8d68-d932e789d4a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" Jul 30 00:51:40.511880 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.511857 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1503a956-5e65-4c0b-8d68-d932e789d4a6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb\" (UID: \"1503a956-5e65-4c0b-8d68-d932e789d4a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" Jul 30 00:51:40.511958 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.511857 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1503a956-5e65-4c0b-8d68-d932e789d4a6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb\" (UID: \"1503a956-5e65-4c0b-8d68-d932e789d4a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" Jul 30 00:51:40.520000 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.519972 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td2p7\" (UniqueName: \"kubernetes.io/projected/1503a956-5e65-4c0b-8d68-d932e789d4a6-kube-api-access-td2p7\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb\" (UID: \"1503a956-5e65-4c0b-8d68-d932e789d4a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" Jul 30 00:51:40.678749 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.678709 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" Jul 30 00:51:40.952874 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.952489 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb"] Jul 30 00:51:40.954420 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.954387 2564 generic.go:356] "Generic (PLEG): container finished" podID="07942485-9a87-436d-8ab9-72797e8e36ae" containerID="aee0fb647a7d868e2a15795c7fe8b867053842b0b23d40557f162b0242d6c9e0" exitCode=0 Jul 30 00:51:40.954552 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.954480 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" event={"ID":"07942485-9a87-436d-8ab9-72797e8e36ae","Type":"ContainerDied","Data":"aee0fb647a7d868e2a15795c7fe8b867053842b0b23d40557f162b0242d6c9e0"} Jul 30 00:51:40.956390 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.956364 2564 generic.go:356] "Generic (PLEG): container finished" podID="721d5439-21ee-46a5-97a8-b34345828dd0" containerID="90adbe257fd466aeb1dd3002cb09983c98d19a38860dd449dba9cfa756dc8963" exitCode=0 Jul 30 00:51:40.956477 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.956447 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" event={"ID":"721d5439-21ee-46a5-97a8-b34345828dd0","Type":"ContainerDied","Data":"90adbe257fd466aeb1dd3002cb09983c98d19a38860dd449dba9cfa756dc8963"} Jul 30 00:51:40.956542 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.956504 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" event={"ID":"721d5439-21ee-46a5-97a8-b34345828dd0","Type":"ContainerStarted","Data":"750ed8d1497e2a62ad49cd82c5fa185045be6428c5657430aa9f762bbee97d0b"} Jul 30 00:51:40.958783 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:40.958758 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" event={"ID":"2c4ece51-398a-41c3-9cd8-87d4cce68a8b","Type":"ContainerStarted","Data":"fa3732d526d26275a93db511df61ad7cb445c6c0ca88065c02faad323b73ca25"} Jul 30 00:51:40.964261 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:51:40.964227 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1503a956_5e65_4c0b_8d68_d932e789d4a6.slice/crio-79f9688d1c53df468b8754bf9eae7572e2ea75b4534d0d004e0d5f045f4560e7 WatchSource:0}: Error finding container 79f9688d1c53df468b8754bf9eae7572e2ea75b4534d0d004e0d5f045f4560e7: Status 404 returned error can't find the container with id 79f9688d1c53df468b8754bf9eae7572e2ea75b4534d0d004e0d5f045f4560e7 Jul 30 00:51:41.966595 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:41.966498 2564 generic.go:356] "Generic (PLEG): container finished" podID="07942485-9a87-436d-8ab9-72797e8e36ae" containerID="9c2993a1f9b9e683ebd69f188f71982b1fa322d7818b864237ad361c2edf6402" exitCode=0 Jul 30 00:51:41.966595 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:41.966569 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" event={"ID":"07942485-9a87-436d-8ab9-72797e8e36ae","Type":"ContainerDied","Data":"9c2993a1f9b9e683ebd69f188f71982b1fa322d7818b864237ad361c2edf6402"} Jul 30 00:51:41.968488 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:41.968462 2564 generic.go:356] "Generic (PLEG): container finished" podID="721d5439-21ee-46a5-97a8-b34345828dd0" containerID="880f2dcf25b32a6d5a6cbd280e206f752815f2eea12e7a0d28858a765c7c5b2d" exitCode=0 Jul 30 00:51:41.968650 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:41.968523 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" event={"ID":"721d5439-21ee-46a5-97a8-b34345828dd0","Type":"ContainerDied","Data":"880f2dcf25b32a6d5a6cbd280e206f752815f2eea12e7a0d28858a765c7c5b2d"} Jul 30 00:51:41.970558 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:41.970533 2564 generic.go:356] "Generic (PLEG): container finished" podID="2c4ece51-398a-41c3-9cd8-87d4cce68a8b" containerID="fa3732d526d26275a93db511df61ad7cb445c6c0ca88065c02faad323b73ca25" exitCode=0 Jul 30 00:51:41.970664 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:41.970615 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" event={"ID":"2c4ece51-398a-41c3-9cd8-87d4cce68a8b","Type":"ContainerDied","Data":"fa3732d526d26275a93db511df61ad7cb445c6c0ca88065c02faad323b73ca25"} Jul 30 00:51:41.972311 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:41.972288 2564 generic.go:356] "Generic (PLEG): container finished" podID="1503a956-5e65-4c0b-8d68-d932e789d4a6" containerID="d38e69cc44e3b4bf3d0985ebb0df57a39637c278ce5175f03f438cf96087246a" exitCode=0 Jul 30 00:51:41.972387 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:41.972322 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" event={"ID":"1503a956-5e65-4c0b-8d68-d932e789d4a6","Type":"ContainerDied","Data":"d38e69cc44e3b4bf3d0985ebb0df57a39637c278ce5175f03f438cf96087246a"} Jul 30 00:51:41.972387 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:41.972351 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" event={"ID":"1503a956-5e65-4c0b-8d68-d932e789d4a6","Type":"ContainerStarted","Data":"79f9688d1c53df468b8754bf9eae7572e2ea75b4534d0d004e0d5f045f4560e7"} Jul 30 00:51:42.980212 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:42.980112 2564 generic.go:356] "Generic (PLEG): container finished" podID="721d5439-21ee-46a5-97a8-b34345828dd0" containerID="1756124e521a1c8ca79afd17a9540f3b819fd2cf19f8043f86bf8f3dfadf4d62" exitCode=0 Jul 30 00:51:42.980212 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:42.980196 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" event={"ID":"721d5439-21ee-46a5-97a8-b34345828dd0","Type":"ContainerDied","Data":"1756124e521a1c8ca79afd17a9540f3b819fd2cf19f8043f86bf8f3dfadf4d62"} Jul 30 00:51:42.982480 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:42.982456 2564 generic.go:356] "Generic (PLEG): container finished" podID="2c4ece51-398a-41c3-9cd8-87d4cce68a8b" containerID="c8caf46328ddfc3604193371850066aef8c829f27e93a1795c1a12c5303709c8" exitCode=0 Jul 30 00:51:42.982606 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:42.982529 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" event={"ID":"2c4ece51-398a-41c3-9cd8-87d4cce68a8b","Type":"ContainerDied","Data":"c8caf46328ddfc3604193371850066aef8c829f27e93a1795c1a12c5303709c8"} Jul 30 00:51:43.119508 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:43.119471 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" Jul 30 00:51:43.241850 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:43.241753 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07942485-9a87-436d-8ab9-72797e8e36ae-util\") pod \"07942485-9a87-436d-8ab9-72797e8e36ae\" (UID: \"07942485-9a87-436d-8ab9-72797e8e36ae\") " Jul 30 00:51:43.241850 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:43.241813 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07942485-9a87-436d-8ab9-72797e8e36ae-bundle\") pod \"07942485-9a87-436d-8ab9-72797e8e36ae\" (UID: \"07942485-9a87-436d-8ab9-72797e8e36ae\") " Jul 30 00:51:43.242107 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:43.241874 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtgtx\" (UniqueName: \"kubernetes.io/projected/07942485-9a87-436d-8ab9-72797e8e36ae-kube-api-access-dtgtx\") pod \"07942485-9a87-436d-8ab9-72797e8e36ae\" (UID: \"07942485-9a87-436d-8ab9-72797e8e36ae\") " Jul 30 00:51:43.242538 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:43.242508 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07942485-9a87-436d-8ab9-72797e8e36ae-bundle" (OuterVolumeSpecName: "bundle") pod "07942485-9a87-436d-8ab9-72797e8e36ae" (UID: "07942485-9a87-436d-8ab9-72797e8e36ae"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:51:43.244246 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:43.244203 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07942485-9a87-436d-8ab9-72797e8e36ae-kube-api-access-dtgtx" (OuterVolumeSpecName: "kube-api-access-dtgtx") pod "07942485-9a87-436d-8ab9-72797e8e36ae" (UID: "07942485-9a87-436d-8ab9-72797e8e36ae"). InnerVolumeSpecName "kube-api-access-dtgtx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:51:43.247936 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:43.247879 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07942485-9a87-436d-8ab9-72797e8e36ae-util" (OuterVolumeSpecName: "util") pod "07942485-9a87-436d-8ab9-72797e8e36ae" (UID: "07942485-9a87-436d-8ab9-72797e8e36ae"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:51:43.342711 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:43.342679 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07942485-9a87-436d-8ab9-72797e8e36ae-bundle\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:51:43.342711 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:43.342704 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtgtx\" (UniqueName: \"kubernetes.io/projected/07942485-9a87-436d-8ab9-72797e8e36ae-kube-api-access-dtgtx\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:51:43.342711 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:43.342716 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07942485-9a87-436d-8ab9-72797e8e36ae-util\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:51:43.989950 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:43.989841 2564 generic.go:356] "Generic (PLEG): container finished" podID="1503a956-5e65-4c0b-8d68-d932e789d4a6" containerID="aac9ee65a5f9998863fb0c84529d82d80f30c58636f58d67f3d7405c4a0376b7" exitCode=0 Jul 30 00:51:43.989950 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:43.989927 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" event={"ID":"1503a956-5e65-4c0b-8d68-d932e789d4a6","Type":"ContainerDied","Data":"aac9ee65a5f9998863fb0c84529d82d80f30c58636f58d67f3d7405c4a0376b7"} Jul 30 00:51:43.991956 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:43.991928 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" event={"ID":"07942485-9a87-436d-8ab9-72797e8e36ae","Type":"ContainerDied","Data":"a2c1d762b39459f051c19735c523da5ea442da1f1ba6169db598e751621ca4bf"} Jul 30 00:51:43.991956 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:43.991952 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p" Jul 30 00:51:43.992081 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:43.991978 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2c1d762b39459f051c19735c523da5ea442da1f1ba6169db598e751621ca4bf" Jul 30 00:51:44.216775 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.216747 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" Jul 30 00:51:44.220183 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.220156 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" Jul 30 00:51:44.351875 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.351835 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/721d5439-21ee-46a5-97a8-b34345828dd0-bundle\") pod \"721d5439-21ee-46a5-97a8-b34345828dd0\" (UID: \"721d5439-21ee-46a5-97a8-b34345828dd0\") " Jul 30 00:51:44.351875 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.351882 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/721d5439-21ee-46a5-97a8-b34345828dd0-util\") pod \"721d5439-21ee-46a5-97a8-b34345828dd0\" (UID: \"721d5439-21ee-46a5-97a8-b34345828dd0\") " Jul 30 00:51:44.352175 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.351941 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-util\") pod \"2c4ece51-398a-41c3-9cd8-87d4cce68a8b\" (UID: \"2c4ece51-398a-41c3-9cd8-87d4cce68a8b\") " Jul 30 00:51:44.352175 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.351968 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ph74\" (UniqueName: \"kubernetes.io/projected/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-kube-api-access-4ph74\") pod \"2c4ece51-398a-41c3-9cd8-87d4cce68a8b\" (UID: \"2c4ece51-398a-41c3-9cd8-87d4cce68a8b\") " Jul 30 00:51:44.352175 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.351992 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzf9m\" (UniqueName: \"kubernetes.io/projected/721d5439-21ee-46a5-97a8-b34345828dd0-kube-api-access-mzf9m\") pod \"721d5439-21ee-46a5-97a8-b34345828dd0\" (UID: \"721d5439-21ee-46a5-97a8-b34345828dd0\") " Jul 30 00:51:44.352175 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.352030 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-bundle\") pod \"2c4ece51-398a-41c3-9cd8-87d4cce68a8b\" (UID: \"2c4ece51-398a-41c3-9cd8-87d4cce68a8b\") " Jul 30 00:51:44.352738 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.352559 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721d5439-21ee-46a5-97a8-b34345828dd0-bundle" (OuterVolumeSpecName: "bundle") pod "721d5439-21ee-46a5-97a8-b34345828dd0" (UID: "721d5439-21ee-46a5-97a8-b34345828dd0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:51:44.352738 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.352556 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-bundle" (OuterVolumeSpecName: "bundle") pod "2c4ece51-398a-41c3-9cd8-87d4cce68a8b" (UID: "2c4ece51-398a-41c3-9cd8-87d4cce68a8b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:51:44.354687 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.354661 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/721d5439-21ee-46a5-97a8-b34345828dd0-kube-api-access-mzf9m" (OuterVolumeSpecName: "kube-api-access-mzf9m") pod "721d5439-21ee-46a5-97a8-b34345828dd0" (UID: "721d5439-21ee-46a5-97a8-b34345828dd0"). InnerVolumeSpecName "kube-api-access-mzf9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:51:44.354687 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.354668 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-kube-api-access-4ph74" (OuterVolumeSpecName: "kube-api-access-4ph74") pod "2c4ece51-398a-41c3-9cd8-87d4cce68a8b" (UID: "2c4ece51-398a-41c3-9cd8-87d4cce68a8b"). InnerVolumeSpecName "kube-api-access-4ph74". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:51:44.357996 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.357969 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-util" (OuterVolumeSpecName: "util") pod "2c4ece51-398a-41c3-9cd8-87d4cce68a8b" (UID: "2c4ece51-398a-41c3-9cd8-87d4cce68a8b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:51:44.358255 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.358231 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721d5439-21ee-46a5-97a8-b34345828dd0-util" (OuterVolumeSpecName: "util") pod "721d5439-21ee-46a5-97a8-b34345828dd0" (UID: "721d5439-21ee-46a5-97a8-b34345828dd0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:51:44.453354 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.453307 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-bundle\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:51:44.453354 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.453347 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/721d5439-21ee-46a5-97a8-b34345828dd0-bundle\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:51:44.453354 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.453359 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/721d5439-21ee-46a5-97a8-b34345828dd0-util\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:51:44.453613 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.453370 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-util\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:51:44.453613 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.453381 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ph74\" (UniqueName: \"kubernetes.io/projected/2c4ece51-398a-41c3-9cd8-87d4cce68a8b-kube-api-access-4ph74\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:51:44.453613 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.453395 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzf9m\" (UniqueName: \"kubernetes.io/projected/721d5439-21ee-46a5-97a8-b34345828dd0-kube-api-access-mzf9m\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:51:44.511423 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.511384 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cc6c6ddc4-674b2"] Jul 30 00:51:44.625289 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.625207 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:44.724133 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.724099 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:44.782127 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.782087 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:44.999709 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.999626 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" Jul 30 00:51:45.000160 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.999620 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc" event={"ID":"721d5439-21ee-46a5-97a8-b34345828dd0","Type":"ContainerDied","Data":"750ed8d1497e2a62ad49cd82c5fa185045be6428c5657430aa9f762bbee97d0b"} Jul 30 00:51:45.000160 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:44.999752 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="750ed8d1497e2a62ad49cd82c5fa185045be6428c5657430aa9f762bbee97d0b" Jul 30 00:51:45.001667 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:45.001646 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" Jul 30 00:51:45.001807 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:45.001675 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks" event={"ID":"2c4ece51-398a-41c3-9cd8-87d4cce68a8b","Type":"ContainerDied","Data":"de5e3c93b6a3b9f499b83f9f60481cb2ea86ef8ba956f055b5b9b8efc0723b5e"} Jul 30 00:51:45.001807 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:45.001718 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de5e3c93b6a3b9f499b83f9f60481cb2ea86ef8ba956f055b5b9b8efc0723b5e" Jul 30 00:51:45.004009 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:45.003981 2564 generic.go:356] "Generic (PLEG): container finished" podID="1503a956-5e65-4c0b-8d68-d932e789d4a6" containerID="920c8e67af316471d9b48aa81be46200b086e87b4e976dfb200878fe07933b95" exitCode=0 Jul 30 00:51:45.004116 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:45.004055 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" event={"ID":"1503a956-5e65-4c0b-8d68-d932e789d4a6","Type":"ContainerDied","Data":"920c8e67af316471d9b48aa81be46200b086e87b4e976dfb200878fe07933b95"} Jul 30 00:51:46.143725 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:46.143695 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" Jul 30 00:51:46.268229 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:46.268122 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1503a956-5e65-4c0b-8d68-d932e789d4a6-bundle\") pod \"1503a956-5e65-4c0b-8d68-d932e789d4a6\" (UID: \"1503a956-5e65-4c0b-8d68-d932e789d4a6\") " Jul 30 00:51:46.268229 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:46.268182 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-td2p7\" (UniqueName: \"kubernetes.io/projected/1503a956-5e65-4c0b-8d68-d932e789d4a6-kube-api-access-td2p7\") pod \"1503a956-5e65-4c0b-8d68-d932e789d4a6\" (UID: \"1503a956-5e65-4c0b-8d68-d932e789d4a6\") " Jul 30 00:51:46.268435 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:46.268278 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1503a956-5e65-4c0b-8d68-d932e789d4a6-util\") pod \"1503a956-5e65-4c0b-8d68-d932e789d4a6\" (UID: \"1503a956-5e65-4c0b-8d68-d932e789d4a6\") " Jul 30 00:51:46.268674 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:46.268645 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1503a956-5e65-4c0b-8d68-d932e789d4a6-bundle" (OuterVolumeSpecName: "bundle") pod "1503a956-5e65-4c0b-8d68-d932e789d4a6" (UID: "1503a956-5e65-4c0b-8d68-d932e789d4a6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:51:46.270424 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:46.270389 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1503a956-5e65-4c0b-8d68-d932e789d4a6-kube-api-access-td2p7" (OuterVolumeSpecName: "kube-api-access-td2p7") pod "1503a956-5e65-4c0b-8d68-d932e789d4a6" (UID: "1503a956-5e65-4c0b-8d68-d932e789d4a6"). InnerVolumeSpecName "kube-api-access-td2p7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:51:46.273343 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:46.273317 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1503a956-5e65-4c0b-8d68-d932e789d4a6-util" (OuterVolumeSpecName: "util") pod "1503a956-5e65-4c0b-8d68-d932e789d4a6" (UID: "1503a956-5e65-4c0b-8d68-d932e789d4a6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:51:46.369243 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:46.369203 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-td2p7\" (UniqueName: \"kubernetes.io/projected/1503a956-5e65-4c0b-8d68-d932e789d4a6-kube-api-access-td2p7\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:51:46.369243 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:46.369237 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1503a956-5e65-4c0b-8d68-d932e789d4a6-util\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:51:46.369243 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:46.369247 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1503a956-5e65-4c0b-8d68-d932e789d4a6-bundle\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:51:47.016400 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:47.016365 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" event={"ID":"1503a956-5e65-4c0b-8d68-d932e789d4a6","Type":"ContainerDied","Data":"79f9688d1c53df468b8754bf9eae7572e2ea75b4534d0d004e0d5f045f4560e7"} Jul 30 00:51:47.016400 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:47.016385 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb" Jul 30 00:51:47.016604 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:47.016412 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79f9688d1c53df468b8754bf9eae7572e2ea75b4534d0d004e0d5f045f4560e7" Jul 30 00:51:51.935164 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:51.935131 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:55.858639 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:55.858605 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:57.565074 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:57.565035 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:59.157439 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:59.157407 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:59.394437 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:59.394406 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:59.543419 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:51:59.543334 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:02.743089 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:02.743053 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:02.996166 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:02.996080 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:03.097845 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:03.097810 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:06.043144 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:06.043108 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:06.142435 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:06.142395 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:06.584073 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:06.584038 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:06.951865 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:06.951774 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:07.052748 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:07.052715 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:09.128753 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.128665 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:09.226487 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.226451 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:09.533193 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.533103 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cc6c6ddc4-674b2" podUID="cd73f666-b565-4f74-a928-d1612dd4e6ef" containerName="console" containerID="cri-o://fcc4b28c4606339f89cca00c59fa5af012f402c6a5d30f824d6076359b1736c2" gracePeriod=15 Jul 30 00:52:09.793173 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.793104 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:52:09.989111 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.989074 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-serving-cert\") pod \"cd73f666-b565-4f74-a928-d1612dd4e6ef\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " Jul 30 00:52:09.989309 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.989149 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-service-ca\") pod \"cd73f666-b565-4f74-a928-d1612dd4e6ef\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " Jul 30 00:52:09.989309 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.989178 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdfth\" (UniqueName: \"kubernetes.io/projected/cd73f666-b565-4f74-a928-d1612dd4e6ef-kube-api-access-bdfth\") pod \"cd73f666-b565-4f74-a928-d1612dd4e6ef\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " Jul 30 00:52:09.989309 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.989194 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-oauth-serving-cert\") pod \"cd73f666-b565-4f74-a928-d1612dd4e6ef\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " Jul 30 00:52:09.989309 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.989264 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-config\") pod \"cd73f666-b565-4f74-a928-d1612dd4e6ef\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " Jul 30 00:52:09.989309 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.989283 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-oauth-config\") pod \"cd73f666-b565-4f74-a928-d1612dd4e6ef\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " Jul 30 00:52:09.989309 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.989301 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-trusted-ca-bundle\") pod \"cd73f666-b565-4f74-a928-d1612dd4e6ef\" (UID: \"cd73f666-b565-4f74-a928-d1612dd4e6ef\") " Jul 30 00:52:09.990170 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.990120 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-config" (OuterVolumeSpecName: "console-config") pod "cd73f666-b565-4f74-a928-d1612dd4e6ef" (UID: "cd73f666-b565-4f74-a928-d1612dd4e6ef"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:52:09.990437 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.989642 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-service-ca" (OuterVolumeSpecName: "service-ca") pod "cd73f666-b565-4f74-a928-d1612dd4e6ef" (UID: "cd73f666-b565-4f74-a928-d1612dd4e6ef"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:52:09.990526 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.989640 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cd73f666-b565-4f74-a928-d1612dd4e6ef" (UID: "cd73f666-b565-4f74-a928-d1612dd4e6ef"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:52:09.990579 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.990524 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cd73f666-b565-4f74-a928-d1612dd4e6ef" (UID: "cd73f666-b565-4f74-a928-d1612dd4e6ef"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:52:09.996582 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.996548 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cd73f666-b565-4f74-a928-d1612dd4e6ef" (UID: "cd73f666-b565-4f74-a928-d1612dd4e6ef"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:52:09.996772 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.996541 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cd73f666-b565-4f74-a928-d1612dd4e6ef" (UID: "cd73f666-b565-4f74-a928-d1612dd4e6ef"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:52:09.998108 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:09.998077 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd73f666-b565-4f74-a928-d1612dd4e6ef-kube-api-access-bdfth" (OuterVolumeSpecName: "kube-api-access-bdfth") pod "cd73f666-b565-4f74-a928-d1612dd4e6ef" (UID: "cd73f666-b565-4f74-a928-d1612dd4e6ef"). InnerVolumeSpecName "kube-api-access-bdfth". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:52:10.091056 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.091018 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-config\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:52:10.091056 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.091048 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-oauth-config\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:52:10.091056 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.091059 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-trusted-ca-bundle\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:52:10.091056 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.091068 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd73f666-b565-4f74-a928-d1612dd4e6ef-console-serving-cert\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:52:10.091336 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.091077 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-service-ca\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:52:10.091336 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.091085 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdfth\" (UniqueName: \"kubernetes.io/projected/cd73f666-b565-4f74-a928-d1612dd4e6ef-kube-api-access-bdfth\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:52:10.091336 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.091094 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd73f666-b565-4f74-a928-d1612dd4e6ef-oauth-serving-cert\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 00:52:10.136444 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.136415 2564 generic.go:356] "Generic (PLEG): container finished" podID="cd73f666-b565-4f74-a928-d1612dd4e6ef" containerID="fcc4b28c4606339f89cca00c59fa5af012f402c6a5d30f824d6076359b1736c2" exitCode=0 Jul 30 00:52:10.136848 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.136478 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc6c6ddc4-674b2" Jul 30 00:52:10.136848 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.136497 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc6c6ddc4-674b2" event={"ID":"cd73f666-b565-4f74-a928-d1612dd4e6ef","Type":"ContainerDied","Data":"fcc4b28c4606339f89cca00c59fa5af012f402c6a5d30f824d6076359b1736c2"} Jul 30 00:52:10.136848 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.136573 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc6c6ddc4-674b2" event={"ID":"cd73f666-b565-4f74-a928-d1612dd4e6ef","Type":"ContainerDied","Data":"1d87330f06166d922e391de849ad4f9b6bde0df2a9d096c012a44fd95847e2fa"} Jul 30 00:52:10.136848 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.136594 2564 scope.go:117] "RemoveContainer" containerID="fcc4b28c4606339f89cca00c59fa5af012f402c6a5d30f824d6076359b1736c2" Jul 30 00:52:10.145434 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.145412 2564 scope.go:117] "RemoveContainer" containerID="fcc4b28c4606339f89cca00c59fa5af012f402c6a5d30f824d6076359b1736c2" Jul 30 00:52:10.145753 ip-10-0-136-247 kubenswrapper[2564]: E0730 00:52:10.145734 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcc4b28c4606339f89cca00c59fa5af012f402c6a5d30f824d6076359b1736c2\": container with ID starting with fcc4b28c4606339f89cca00c59fa5af012f402c6a5d30f824d6076359b1736c2 not found: ID does not exist" containerID="fcc4b28c4606339f89cca00c59fa5af012f402c6a5d30f824d6076359b1736c2" Jul 30 00:52:10.145800 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.145762 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcc4b28c4606339f89cca00c59fa5af012f402c6a5d30f824d6076359b1736c2"} err="failed to get container status \"fcc4b28c4606339f89cca00c59fa5af012f402c6a5d30f824d6076359b1736c2\": rpc error: code = NotFound desc = could not find container \"fcc4b28c4606339f89cca00c59fa5af012f402c6a5d30f824d6076359b1736c2\": container with ID starting with fcc4b28c4606339f89cca00c59fa5af012f402c6a5d30f824d6076359b1736c2 not found: ID does not exist" Jul 30 00:52:10.158050 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.158013 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cc6c6ddc4-674b2"] Jul 30 00:52:10.161519 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:10.161492 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cc6c6ddc4-674b2"] Jul 30 00:52:11.670565 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:11.670526 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd73f666-b565-4f74-a928-d1612dd4e6ef" path="/var/lib/kubelet/pods/cd73f666-b565-4f74-a928-d1612dd4e6ef/volumes" Jul 30 00:52:14.555821 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:14.555783 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:14.656578 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:14.656546 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:20.355750 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:20.355713 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:20.456207 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:20.456170 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:23.370734 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:23.370706 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:23.471411 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:23.471375 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:27.159131 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:27.159094 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:27.215908 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:27.215870 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:34.294504 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:34.294461 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:34.385408 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:34.385378 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:40.184784 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:40.184741 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:40.285136 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:40.285090 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:40.403209 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:40.403172 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:43.182978 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:43.182935 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:43.283279 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:43.283211 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:45.009919 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.009864 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc"] Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010279 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1503a956-5e65-4c0b-8d68-d932e789d4a6" containerName="util" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010294 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1503a956-5e65-4c0b-8d68-d932e789d4a6" containerName="util" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010307 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd73f666-b565-4f74-a928-d1612dd4e6ef" containerName="console" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010312 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd73f666-b565-4f74-a928-d1612dd4e6ef" containerName="console" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010319 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="721d5439-21ee-46a5-97a8-b34345828dd0" containerName="util" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010324 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="721d5439-21ee-46a5-97a8-b34345828dd0" containerName="util" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010337 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c4ece51-398a-41c3-9cd8-87d4cce68a8b" containerName="util" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010342 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c4ece51-398a-41c3-9cd8-87d4cce68a8b" containerName="util" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010349 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1503a956-5e65-4c0b-8d68-d932e789d4a6" containerName="pull" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010354 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1503a956-5e65-4c0b-8d68-d932e789d4a6" containerName="pull" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010361 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1503a956-5e65-4c0b-8d68-d932e789d4a6" containerName="extract" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010365 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1503a956-5e65-4c0b-8d68-d932e789d4a6" containerName="extract" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010375 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c4ece51-398a-41c3-9cd8-87d4cce68a8b" containerName="extract" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010380 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c4ece51-398a-41c3-9cd8-87d4cce68a8b" containerName="extract" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010388 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07942485-9a87-436d-8ab9-72797e8e36ae" containerName="util" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010393 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="07942485-9a87-436d-8ab9-72797e8e36ae" containerName="util" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010398 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="721d5439-21ee-46a5-97a8-b34345828dd0" containerName="pull" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010403 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="721d5439-21ee-46a5-97a8-b34345828dd0" containerName="pull" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010409 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07942485-9a87-436d-8ab9-72797e8e36ae" containerName="pull" Jul 30 00:52:45.010404 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010415 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="07942485-9a87-436d-8ab9-72797e8e36ae" containerName="pull" Jul 30 00:52:45.011428 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010421 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07942485-9a87-436d-8ab9-72797e8e36ae" containerName="extract" Jul 30 00:52:45.011428 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010427 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="07942485-9a87-436d-8ab9-72797e8e36ae" containerName="extract" Jul 30 00:52:45.011428 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010432 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="721d5439-21ee-46a5-97a8-b34345828dd0" containerName="extract" Jul 30 00:52:45.011428 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010437 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="721d5439-21ee-46a5-97a8-b34345828dd0" containerName="extract" Jul 30 00:52:45.011428 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010444 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c4ece51-398a-41c3-9cd8-87d4cce68a8b" containerName="pull" Jul 30 00:52:45.011428 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010449 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c4ece51-398a-41c3-9cd8-87d4cce68a8b" containerName="pull" Jul 30 00:52:45.011428 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010505 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1503a956-5e65-4c0b-8d68-d932e789d4a6" containerName="extract" Jul 30 00:52:45.011428 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010511 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="721d5439-21ee-46a5-97a8-b34345828dd0" containerName="extract" Jul 30 00:52:45.011428 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010519 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd73f666-b565-4f74-a928-d1612dd4e6ef" containerName="console" Jul 30 00:52:45.011428 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010525 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c4ece51-398a-41c3-9cd8-87d4cce68a8b" containerName="extract" Jul 30 00:52:45.011428 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.010531 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="07942485-9a87-436d-8ab9-72797e8e36ae" containerName="extract" Jul 30 00:52:45.013766 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.013731 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.016195 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.016170 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-nch4t\"" Jul 30 00:52:45.024970 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.024942 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc"] Jul 30 00:52:45.047017 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.046969 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:45.089946 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.089893 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba299a42-e1c8-4551-a790-da168ca7f972-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.089946 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.089950 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba299a42-e1c8-4551-a790-da168ca7f972-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.090160 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.089974 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba299a42-e1c8-4551-a790-da168ca7f972-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.090160 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.090026 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba299a42-e1c8-4551-a790-da168ca7f972-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.090160 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.090072 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba299a42-e1c8-4551-a790-da168ca7f972-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.090160 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.090119 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba299a42-e1c8-4551-a790-da168ca7f972-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.090160 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.090150 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtknw\" (UniqueName: \"kubernetes.io/projected/ba299a42-e1c8-4551-a790-da168ca7f972-kube-api-access-xtknw\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.090341 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.090183 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba299a42-e1c8-4551-a790-da168ca7f972-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.090341 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.090218 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba299a42-e1c8-4551-a790-da168ca7f972-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.191130 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.191095 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba299a42-e1c8-4551-a790-da168ca7f972-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.191130 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.191134 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba299a42-e1c8-4551-a790-da168ca7f972-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.191429 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.191155 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtknw\" (UniqueName: \"kubernetes.io/projected/ba299a42-e1c8-4551-a790-da168ca7f972-kube-api-access-xtknw\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.191429 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.191182 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba299a42-e1c8-4551-a790-da168ca7f972-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.191429 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.191216 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba299a42-e1c8-4551-a790-da168ca7f972-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.191429 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.191262 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba299a42-e1c8-4551-a790-da168ca7f972-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.191429 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.191293 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba299a42-e1c8-4551-a790-da168ca7f972-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.191429 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.191320 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba299a42-e1c8-4551-a790-da168ca7f972-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.191429 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.191371 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba299a42-e1c8-4551-a790-da168ca7f972-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.191758 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.191710 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba299a42-e1c8-4551-a790-da168ca7f972-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.191758 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.191724 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba299a42-e1c8-4551-a790-da168ca7f972-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.191948 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.191909 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba299a42-e1c8-4551-a790-da168ca7f972-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.192061 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.191979 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba299a42-e1c8-4551-a790-da168ca7f972-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.192061 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.192021 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba299a42-e1c8-4551-a790-da168ca7f972-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.193681 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.193654 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba299a42-e1c8-4551-a790-da168ca7f972-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.193925 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.193893 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba299a42-e1c8-4551-a790-da168ca7f972-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.199102 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.199068 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba299a42-e1c8-4551-a790-da168ca7f972-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.199231 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.199185 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtknw\" (UniqueName: \"kubernetes.io/projected/ba299a42-e1c8-4551-a790-da168ca7f972-kube-api-access-xtknw\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jffhc\" (UID: \"ba299a42-e1c8-4551-a790-da168ca7f972\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.326149 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.326102 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:45.472340 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.472298 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:45.473828 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.473802 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc"] Jul 30 00:52:45.475701 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:52:45.475676 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba299a42_e1c8_4551_a790_da168ca7f972.slice/crio-7d8b1176eee56eade63ee5a1d4e28d3a624e718f4292157509d4f37e876a87ca WatchSource:0}: Error finding container 7d8b1176eee56eade63ee5a1d4e28d3a624e718f4292157509d4f37e876a87ca: Status 404 returned error can't find the container with id 7d8b1176eee56eade63ee5a1d4e28d3a624e718f4292157509d4f37e876a87ca Jul 30 00:52:45.478017 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.477981 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 00:52:45.478094 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.478072 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 00:52:45.478135 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:45.478115 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 00:52:46.328108 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:46.328063 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" event={"ID":"ba299a42-e1c8-4551-a790-da168ca7f972","Type":"ContainerStarted","Data":"3013c93ac0f6b3cb9f9259f9ccc0536fa91ddef354f014972a9c0a709aab21ad"} Jul 30 00:52:46.328504 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:46.328131 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" event={"ID":"ba299a42-e1c8-4551-a790-da168ca7f972","Type":"ContainerStarted","Data":"7d8b1176eee56eade63ee5a1d4e28d3a624e718f4292157509d4f37e876a87ca"} Jul 30 00:52:46.349422 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:46.349365 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" podStartSLOduration=2.349346106 podStartE2EDuration="2.349346106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:52:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:52:46.347353915 +0000 UTC m=+569.148311026" watchObservedRunningTime="2026-07-30 00:52:46.349346106 +0000 UTC m=+569.150303212" Jul 30 00:52:47.326730 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:47.326688 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:47.332547 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:47.332521 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:47.333646 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:47.333629 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:47.334398 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:47.334382 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:52:47.347049 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:47.347017 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:47.420886 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:47.420857 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:47.525365 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:47.525302 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:48.228600 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:48.228548 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:48.329641 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:48.329592 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:54.596139 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:54.596098 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:54.648960 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:52:54.648930 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:00.401604 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:00.401565 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:00.882753 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:00.882719 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:03.423946 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:03.423910 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:03.521707 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:03.521669 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:03.853154 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:03.853115 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:05.313829 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:05.313787 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:05.413769 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:05.413729 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:16.255271 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:16.255228 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:16.554487 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:16.554388 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:22.450853 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:22.450810 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:23.040189 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:23.040157 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:23.114842 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:23.114806 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:26.519335 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:26.519292 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:26.629275 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:26.629239 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:57.715288 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:57.715248 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:59.187789 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:59.187756 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:59.370978 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:59.370943 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:59.607637 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:59.607599 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:59.972254 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:53:59.972163 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:00.078081 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:00.078043 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:07.752651 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:07.752613 2564 generic.go:356] "Generic (PLEG): container finished" podID="ba299a42-e1c8-4551-a790-da168ca7f972" containerID="3013c93ac0f6b3cb9f9259f9ccc0536fa91ddef354f014972a9c0a709aab21ad" exitCode=0 Jul 30 00:54:07.753145 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:07.752668 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" event={"ID":"ba299a42-e1c8-4551-a790-da168ca7f972","Type":"ContainerDied","Data":"3013c93ac0f6b3cb9f9259f9ccc0536fa91ddef354f014972a9c0a709aab21ad"} Jul 30 00:54:07.753145 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:07.753122 2564 scope.go:117] "RemoveContainer" containerID="3013c93ac0f6b3cb9f9259f9ccc0536fa91ddef354f014972a9c0a709aab21ad" Jul 30 00:54:07.754202 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:07.754167 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 00:54:07.754315 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:07.754263 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 00:54:07.754391 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:07.754314 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 00:54:08.091643 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:08.091614 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:08.759288 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:08.759252 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" event={"ID":"ba299a42-e1c8-4551-a790-da168ca7f972","Type":"ContainerStarted","Data":"b9eb445c38aaf08fbc62ce6fc39df9ed0a4924394a5ea8a70fdff4dd934ffcc3"} Jul 30 00:54:09.326548 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:09.326500 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:54:09.326840 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:09.326815 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jffhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 30 00:54:09.326945 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:09.326876 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" podUID="ba299a42-e1c8-4551-a790-da168ca7f972" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 30 00:54:10.217647 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:10.217598 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:10.320653 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:10.320617 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:10.327555 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:10.327511 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jffhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 30 00:54:10.327760 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:10.327577 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" podUID="ba299a42-e1c8-4551-a790-da168ca7f972" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 30 00:54:11.326757 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:11.326719 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jffhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 30 00:54:11.327179 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:11.326834 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" podUID="ba299a42-e1c8-4551-a790-da168ca7f972" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 30 00:54:12.327462 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:12.327418 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jffhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 30 00:54:12.327880 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:12.327483 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" podUID="ba299a42-e1c8-4551-a790-da168ca7f972" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 30 00:54:13.331349 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:13.331318 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:54:13.331734 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:13.331626 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:54:13.332507 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:13.332491 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jffhc" Jul 30 00:54:24.370187 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:24.370147 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:26.578432 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:26.578398 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:40.948158 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:40.948119 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q"] Jul 30 00:54:40.951994 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:40.951973 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:40.954314 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:40.954287 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 30 00:54:40.955189 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:40.955174 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 30 00:54:40.955348 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:40.955326 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-stf8p\"" Jul 30 00:54:40.955411 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:40.955401 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 30 00:54:40.959602 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:40.959572 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q"] Jul 30 00:54:40.973546 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:40.973505 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:41.085266 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.085231 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5a29493-9c81-4c16-a645-c961b8f1820b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.085447 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.085298 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bdvs\" (UniqueName: \"kubernetes.io/projected/d5a29493-9c81-4c16-a645-c961b8f1820b-kube-api-access-6bdvs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.085447 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.085320 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5a29493-9c81-4c16-a645-c961b8f1820b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.085447 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.085381 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5a29493-9c81-4c16-a645-c961b8f1820b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.085447 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.085401 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5a29493-9c81-4c16-a645-c961b8f1820b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.085447 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.085421 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5a29493-9c81-4c16-a645-c961b8f1820b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.186317 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.186256 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5a29493-9c81-4c16-a645-c961b8f1820b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.186534 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.186350 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5a29493-9c81-4c16-a645-c961b8f1820b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.186534 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.186384 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5a29493-9c81-4c16-a645-c961b8f1820b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.186534 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.186413 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5a29493-9c81-4c16-a645-c961b8f1820b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.186534 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.186466 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5a29493-9c81-4c16-a645-c961b8f1820b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.186752 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.186539 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bdvs\" (UniqueName: \"kubernetes.io/projected/d5a29493-9c81-4c16-a645-c961b8f1820b-kube-api-access-6bdvs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.186805 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.186767 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5a29493-9c81-4c16-a645-c961b8f1820b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.186853 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.186813 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5a29493-9c81-4c16-a645-c961b8f1820b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.186853 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.186840 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5a29493-9c81-4c16-a645-c961b8f1820b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.188655 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.188635 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5a29493-9c81-4c16-a645-c961b8f1820b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.188950 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.188928 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5a29493-9c81-4c16-a645-c961b8f1820b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.194758 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.194730 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bdvs\" (UniqueName: \"kubernetes.io/projected/d5a29493-9c81-4c16-a645-c961b8f1820b-kube-api-access-6bdvs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qcc4q\" (UID: \"d5a29493-9c81-4c16-a645-c961b8f1820b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.264472 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.264375 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:41.406195 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.406124 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q"] Jul 30 00:54:41.408744 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:54:41.408715 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5a29493_9c81_4c16_a645_c961b8f1820b.slice/crio-887a0923f0a25fd8468da5dbf4dada1f3d4b6d4c34908a9160fe98cd46c45f0a WatchSource:0}: Error finding container 887a0923f0a25fd8468da5dbf4dada1f3d4b6d4c34908a9160fe98cd46c45f0a: Status 404 returned error can't find the container with id 887a0923f0a25fd8468da5dbf4dada1f3d4b6d4c34908a9160fe98cd46c45f0a Jul 30 00:54:41.410500 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.410482 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 00:54:41.936987 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:41.936939 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" event={"ID":"d5a29493-9c81-4c16-a645-c961b8f1820b","Type":"ContainerStarted","Data":"887a0923f0a25fd8468da5dbf4dada1f3d4b6d4c34908a9160fe98cd46c45f0a"} Jul 30 00:54:42.943646 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:42.943606 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" event={"ID":"d5a29493-9c81-4c16-a645-c961b8f1820b","Type":"ContainerStarted","Data":"5ed85c278997f774901b0d5053bbfc5d4e1ebfa4c18c229f0257b06927d7ee51"} Jul 30 00:54:42.960355 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:42.960298 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" podStartSLOduration=1.9884119519999999 podStartE2EDuration="2.960280738s" totalImagesPullingTime="971.868786ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:54:42.959114052 +0000 UTC m=+685.760071161" watchObservedRunningTime="2026-07-30 00:54:42.960280738 +0000 UTC m=+685.761237843" Jul 30 00:54:50.542067 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.542030 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh"] Jul 30 00:54:50.545728 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.545711 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.547940 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.547921 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 30 00:54:50.556067 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.556044 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh"] Jul 30 00:54:50.649237 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.649203 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:50.675247 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.675209 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49a1d67b-a216-4fdf-a167-e7218161c8c9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.675421 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.675255 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n798\" (UniqueName: \"kubernetes.io/projected/49a1d67b-a216-4fdf-a167-e7218161c8c9-kube-api-access-7n798\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.675421 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.675277 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49a1d67b-a216-4fdf-a167-e7218161c8c9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.675421 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.675295 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49a1d67b-a216-4fdf-a167-e7218161c8c9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.675421 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.675362 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49a1d67b-a216-4fdf-a167-e7218161c8c9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.675421 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.675392 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49a1d67b-a216-4fdf-a167-e7218161c8c9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.776085 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.776048 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49a1d67b-a216-4fdf-a167-e7218161c8c9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.776266 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.776095 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n798\" (UniqueName: \"kubernetes.io/projected/49a1d67b-a216-4fdf-a167-e7218161c8c9-kube-api-access-7n798\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.776266 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.776148 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49a1d67b-a216-4fdf-a167-e7218161c8c9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.776266 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.776184 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49a1d67b-a216-4fdf-a167-e7218161c8c9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.776266 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.776223 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49a1d67b-a216-4fdf-a167-e7218161c8c9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.776266 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.776252 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49a1d67b-a216-4fdf-a167-e7218161c8c9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.776614 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.776586 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49a1d67b-a216-4fdf-a167-e7218161c8c9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.776677 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.776594 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49a1d67b-a216-4fdf-a167-e7218161c8c9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.776775 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.776755 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49a1d67b-a216-4fdf-a167-e7218161c8c9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.778495 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.778471 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49a1d67b-a216-4fdf-a167-e7218161c8c9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.778745 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.778724 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49a1d67b-a216-4fdf-a167-e7218161c8c9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.784023 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.783995 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n798\" (UniqueName: \"kubernetes.io/projected/49a1d67b-a216-4fdf-a167-e7218161c8c9-kube-api-access-7n798\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2lgmh\" (UID: \"49a1d67b-a216-4fdf-a167-e7218161c8c9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.856483 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.856443 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:54:50.987808 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:50.987784 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh"] Jul 30 00:54:50.989577 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:54:50.989550 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49a1d67b_a216_4fdf_a167_e7218161c8c9.slice/crio-b92bb7b764f4f6bf9a2c88473c317c0cb0e6f4c8f3afce20cfb046dd1bd9bb2d WatchSource:0}: Error finding container b92bb7b764f4f6bf9a2c88473c317c0cb0e6f4c8f3afce20cfb046dd1bd9bb2d: Status 404 returned error can't find the container with id b92bb7b764f4f6bf9a2c88473c317c0cb0e6f4c8f3afce20cfb046dd1bd9bb2d Jul 30 00:54:51.264982 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:51.264955 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:51.265102 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:51.265016 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:51.278473 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:51.278441 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:51.991627 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:51.991591 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" event={"ID":"49a1d67b-a216-4fdf-a167-e7218161c8c9","Type":"ContainerStarted","Data":"db6ac5c19b8a08e86e6aebf30091ebfdb49816999293fe76299634c01336e3a8"} Jul 30 00:54:51.992053 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:51.991644 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" event={"ID":"49a1d67b-a216-4fdf-a167-e7218161c8c9","Type":"ContainerStarted","Data":"b92bb7b764f4f6bf9a2c88473c317c0cb0e6f4c8f3afce20cfb046dd1bd9bb2d"} Jul 30 00:54:52.003134 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:52.003101 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qcc4q" Jul 30 00:54:52.010327 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:54:52.010282 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" podStartSLOduration=1.803127812 podStartE2EDuration="2.010267213s" totalImagesPullingTime="207.139401ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:54:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:54:52.009602196 +0000 UTC m=+694.810559303" watchObservedRunningTime="2026-07-30 00:54:52.010267213 +0000 UTC m=+694.811224354" Jul 30 00:55:00.857098 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:00.857047 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:55:00.857505 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:00.857124 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:55:00.869778 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:00.869749 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:55:01.048115 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:01.048085 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2lgmh" Jul 30 00:55:11.355309 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:11.355218 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:55:11.676323 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:11.676242 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:55:19.261275 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:19.261227 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:55:27.441251 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.441214 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk"] Jul 30 00:55:27.445050 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.445021 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.447322 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.447298 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 30 00:55:27.454023 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.453994 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk"] Jul 30 00:55:27.469261 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.469231 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:55:27.529868 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.529825 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b629936-5f15-4329-b291-daa5cdff78f3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.529868 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.529870 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b629936-5f15-4329-b291-daa5cdff78f3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.530180 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.529972 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b629936-5f15-4329-b291-daa5cdff78f3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.530180 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.530019 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b629936-5f15-4329-b291-daa5cdff78f3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.530180 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.530046 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh2bt\" (UniqueName: \"kubernetes.io/projected/0b629936-5f15-4329-b291-daa5cdff78f3-kube-api-access-dh2bt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.530180 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.530088 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b629936-5f15-4329-b291-daa5cdff78f3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.631054 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.631013 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b629936-5f15-4329-b291-daa5cdff78f3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.631054 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.631052 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b629936-5f15-4329-b291-daa5cdff78f3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.631328 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.631094 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b629936-5f15-4329-b291-daa5cdff78f3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.631328 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.631137 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b629936-5f15-4329-b291-daa5cdff78f3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.631328 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.631162 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh2bt\" (UniqueName: \"kubernetes.io/projected/0b629936-5f15-4329-b291-daa5cdff78f3-kube-api-access-dh2bt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.631328 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.631202 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b629936-5f15-4329-b291-daa5cdff78f3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.631540 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.631507 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b629936-5f15-4329-b291-daa5cdff78f3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.631592 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.631546 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b629936-5f15-4329-b291-daa5cdff78f3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.631628 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.631589 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b629936-5f15-4329-b291-daa5cdff78f3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.633642 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.633606 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b629936-5f15-4329-b291-daa5cdff78f3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.633835 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.633816 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b629936-5f15-4329-b291-daa5cdff78f3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.640249 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.640213 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh2bt\" (UniqueName: \"kubernetes.io/projected/0b629936-5f15-4329-b291-daa5cdff78f3-kube-api-access-dh2bt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwmjk\" (UID: \"0b629936-5f15-4329-b291-daa5cdff78f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.756555 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.756456 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:27.899821 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:27.899790 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk"] Jul 30 00:55:27.901437 ip-10-0-136-247 kubenswrapper[2564]: W0730 00:55:27.901409 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b629936_5f15_4329_b291_daa5cdff78f3.slice/crio-35742d5a755baff8ac68cdb9485cb094f8465c27bb7a4f65115416a59c026baf WatchSource:0}: Error finding container 35742d5a755baff8ac68cdb9485cb094f8465c27bb7a4f65115416a59c026baf: Status 404 returned error can't find the container with id 35742d5a755baff8ac68cdb9485cb094f8465c27bb7a4f65115416a59c026baf Jul 30 00:55:28.181681 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:28.181639 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" event={"ID":"0b629936-5f15-4329-b291-daa5cdff78f3","Type":"ContainerStarted","Data":"288eee85363efb7cbbc5ff07da18b7b627a53b2bf0aa9fbc24fd8294f4aad9cf"} Jul 30 00:55:28.181885 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:28.181697 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" event={"ID":"0b629936-5f15-4329-b291-daa5cdff78f3","Type":"ContainerStarted","Data":"35742d5a755baff8ac68cdb9485cb094f8465c27bb7a4f65115416a59c026baf"} Jul 30 00:55:28.199514 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:28.199463 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" podStartSLOduration=1.020647174 podStartE2EDuration="1.199445373s" totalImagesPullingTime="178.798199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:55:28.198878971 +0000 UTC m=+730.999836077" watchObservedRunningTime="2026-07-30 00:55:28.199445373 +0000 UTC m=+731.000402480" Jul 30 00:55:30.371725 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:30.371692 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:55:37.757032 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:37.756976 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:37.757520 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:37.757058 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:37.769871 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:37.769842 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:55:38.243410 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:55:38.243381 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwmjk" Jul 30 00:56:24.583461 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:56:24.583418 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:56:24.688289 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:56:24.688249 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:08.294981 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:57:08.294942 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:08.396503 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:57:08.396442 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:53.383494 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:57:53.383437 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:29.009553 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:58:29.009511 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:30.251540 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:58:30.251504 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:29.875699 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:59:29.875656 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:34.675357 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:59:34.675323 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:34.725820 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:59:34.725790 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:40.062732 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:59:40.062622 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:40.162196 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:59:40.162160 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:41.424884 ip-10-0-136-247 kubenswrapper[2564]: I0730 00:59:41.424846 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:05:06.684400 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:05:06.684358 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:05:07.973916 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:05:07.973872 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:06:35.047214 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:06:35.047174 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:55.857431 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:09:55.857391 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:10:10.149684 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:10:10.149592 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:10:15.893979 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:10:15.893943 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:10:16.237867 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:10:16.237776 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:10:16.335663 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:10:16.335627 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:10:28.271578 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:10:28.271537 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:10:49.634064 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:10:49.634022 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:10:50.754646 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:10:50.754599 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:22.534550 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:12:22.534504 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:14:47.565425 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:14:47.565337 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:23.914389 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:23.914355 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:26.943866 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:26.943827 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:27.042382 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:27.042344 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:27.379807 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:27.379773 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:33.451479 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:33.451418 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:33.584257 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:33.584224 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:33.690938 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:33.690890 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:33.962301 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:33.962264 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k"] Jul 30 01:15:33.966104 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:33.966075 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:33.968380 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:33.968358 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-76aaf06c-openshift-default-dockercfg-5wzp9\"" Jul 30 01:15:33.980869 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:33.980828 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k"] Jul 30 01:15:34.003033 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.002993 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:34.040139 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.040096 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a715981-3d30-469a-ba92-828fce07a2e4-istio-podinfo\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.040336 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.040160 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-istio-envoy\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.040336 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.040187 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-workload-socket\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.040336 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.040247 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2sh6\" (UniqueName: \"kubernetes.io/projected/3a715981-3d30-469a-ba92-828fce07a2e4-kube-api-access-g2sh6\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.040505 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.040342 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a715981-3d30-469a-ba92-828fce07a2e4-istio-token\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.040505 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.040392 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a715981-3d30-469a-ba92-828fce07a2e4-istiod-ca-cert\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.040505 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.040484 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-workload-certs\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.040624 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.040538 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-credential-socket\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.040624 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.040564 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-istio-data\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.141185 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.141149 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-workload-socket\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.141185 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.141191 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2sh6\" (UniqueName: \"kubernetes.io/projected/3a715981-3d30-469a-ba92-828fce07a2e4-kube-api-access-g2sh6\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.141456 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.141219 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a715981-3d30-469a-ba92-828fce07a2e4-istio-token\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.141456 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.141359 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a715981-3d30-469a-ba92-828fce07a2e4-istiod-ca-cert\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.141542 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.141459 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-workload-certs\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.141542 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.141505 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-credential-socket\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.141542 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.141529 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-istio-data\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.141687 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.141557 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a715981-3d30-469a-ba92-828fce07a2e4-istio-podinfo\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.141687 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.141633 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-workload-socket\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.141687 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.141634 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-istio-envoy\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.141867 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.141845 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-workload-certs\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.141951 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.141885 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-credential-socket\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.142002 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.141952 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-istio-data\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.142188 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.142169 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a715981-3d30-469a-ba92-828fce07a2e4-istiod-ca-cert\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.144198 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.144149 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-istio-envoy\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.146102 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.146079 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a715981-3d30-469a-ba92-828fce07a2e4-istio-podinfo\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.151534 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.151494 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2sh6\" (UniqueName: \"kubernetes.io/projected/3a715981-3d30-469a-ba92-828fce07a2e4-kube-api-access-g2sh6\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.151954 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.151930 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a715981-3d30-469a-ba92-828fce07a2e4-istio-token\") pod \"e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.186100 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.186061 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:34.277435 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.277329 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:34.427984 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.427951 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k"] Jul 30 01:15:34.429596 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:15:34.429567 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a715981_3d30_469a_ba92_828fce07a2e4.slice/crio-b9668518871ff524c39466cb04d63b2daa0adb7454c3a7d280a75902028ae85f WatchSource:0}: Error finding container b9668518871ff524c39466cb04d63b2daa0adb7454c3a7d280a75902028ae85f: Status 404 returned error can't find the container with id b9668518871ff524c39466cb04d63b2daa0adb7454c3a7d280a75902028ae85f Jul 30 01:15:34.432130 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.432095 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:15:34.432222 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.432184 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:15:34.432262 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.432225 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:15:34.669329 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.669280 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" event={"ID":"3a715981-3d30-469a-ba92-828fce07a2e4","Type":"ContainerStarted","Data":"ef5af0138a88fb022aec36cdf7405ca55dcc3481e73cfac6d05f12b594f4b816"} Jul 30 01:15:34.669737 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.669350 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" event={"ID":"3a715981-3d30-469a-ba92-828fce07a2e4","Type":"ContainerStarted","Data":"b9668518871ff524c39466cb04d63b2daa0adb7454c3a7d280a75902028ae85f"} Jul 30 01:15:34.696648 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:34.696591 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" podStartSLOduration=1.696570655 podStartE2EDuration="1.696570655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:15:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:15:34.693393356 +0000 UTC m=+1937.494350463" watchObservedRunningTime="2026-07-30 01:15:34.696570655 +0000 UTC m=+1937.497527838" Jul 30 01:15:35.277972 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:35.277933 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:35.283083 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:35.283052 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:35.674183 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:35.674148 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:35.675198 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:35.675180 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:36.076542 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:36.076512 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:36.172120 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:36.172087 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:37.220923 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:37.220863 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:37.594971 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:37.594938 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:37.833625 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:37.833588 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:37.932819 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:37.932724 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:15:51.716115 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:51.716064 2564 patch_prober.go:28] interesting pod/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 30 01:15:51.716513 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:51.716137 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" podUID="3a715981-3d30-469a-ba92-828fce07a2e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 30 01:15:52.767084 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:52.767054 2564 generic.go:356] "Generic (PLEG): container finished" podID="3a715981-3d30-469a-ba92-828fce07a2e4" containerID="ef5af0138a88fb022aec36cdf7405ca55dcc3481e73cfac6d05f12b594f4b816" exitCode=0 Jul 30 01:15:52.767520 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:52.767131 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" event={"ID":"3a715981-3d30-469a-ba92-828fce07a2e4","Type":"ContainerDied","Data":"ef5af0138a88fb022aec36cdf7405ca55dcc3481e73cfac6d05f12b594f4b816"} Jul 30 01:15:52.767572 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:52.767558 2564 scope.go:117] "RemoveContainer" containerID="ef5af0138a88fb022aec36cdf7405ca55dcc3481e73cfac6d05f12b594f4b816" Jul 30 01:15:52.768701 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:52.768671 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:15:52.768769 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:52.768750 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:15:52.768805 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:52.768781 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:15:53.774067 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:53.774029 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" event={"ID":"3a715981-3d30-469a-ba92-828fce07a2e4","Type":"ContainerStarted","Data":"1d4f87680c472346083a36056d5733b504fa1651881dfa3faae0ffdd6955c702"} Jul 30 01:15:54.278001 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:54.277951 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:54.278224 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:54.278199 2564 patch_prober.go:28] interesting pod/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 30 01:15:54.278268 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:54.278250 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" podUID="3a715981-3d30-469a-ba92-828fce07a2e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 30 01:15:55.278016 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:55.277973 2564 patch_prober.go:28] interesting pod/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 30 01:15:55.278503 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:55.278040 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" podUID="3a715981-3d30-469a-ba92-828fce07a2e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 30 01:15:56.277990 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:56.277948 2564 patch_prober.go:28] interesting pod/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 30 01:15:56.278175 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:56.278005 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" podUID="3a715981-3d30-469a-ba92-828fce07a2e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 30 01:15:57.278571 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:57.278529 2564 patch_prober.go:28] interesting pod/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 30 01:15:57.279013 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:57.278593 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" podUID="3a715981-3d30-469a-ba92-828fce07a2e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 30 01:15:58.281465 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:58.281431 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:58.281909 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:58.281703 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:15:58.282516 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:15:58.282482 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:16:07.696127 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:07.696095 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:07.769432 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:07.769399 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:08.037077 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:08.036977 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k"] Jul 30 01:16:08.037564 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:08.037514 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" podUID="3a715981-3d30-469a-ba92-828fce07a2e4" containerName="istio-proxy" containerID="cri-o://1d4f87680c472346083a36056d5733b504fa1651881dfa3faae0ffdd6955c702" gracePeriod=30 Jul 30 01:16:08.394779 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:08.394746 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:08.575518 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:08.575480 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:08.742250 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:08.742114 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:08.827443 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:08.827388 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:09.011959 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.011860 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd"] Jul 30 01:16:09.015501 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.015481 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.017772 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.017750 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-a6605e44-gw-openshift-default-dockercfg-sbbl7\"" Jul 30 01:16:09.029060 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.029031 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd"] Jul 30 01:16:09.055531 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.055495 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:09.098622 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.098585 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-credential-socket\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.098809 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.098720 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-podinfo\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.098809 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.098771 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-workload-socket\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.099024 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.098814 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-envoy\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.099024 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.098865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fzjp\" (UniqueName: \"kubernetes.io/projected/f55ee3da-277b-4e44-9301-dd396b0a83d1-kube-api-access-4fzjp\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.099024 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.098936 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-workload-certs\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.099203 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.099011 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-data\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.099203 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.099074 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-token\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.099203 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.099103 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f55ee3da-277b-4e44-9301-dd396b0a83d1-istiod-ca-cert\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.200115 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.200075 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-podinfo\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.200115 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.200119 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-workload-socket\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.200381 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.200142 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-envoy\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.200381 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.200163 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fzjp\" (UniqueName: \"kubernetes.io/projected/f55ee3da-277b-4e44-9301-dd396b0a83d1-kube-api-access-4fzjp\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.200381 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.200201 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-workload-certs\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.200381 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.200247 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-data\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.200381 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.200280 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-token\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.200381 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.200304 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f55ee3da-277b-4e44-9301-dd396b0a83d1-istiod-ca-cert\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.200381 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.200346 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-credential-socket\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.200743 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.200592 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-workload-socket\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.200743 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.200660 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-data\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.200845 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.200760 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-workload-certs\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.200845 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.200806 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-credential-socket\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.201259 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.201238 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f55ee3da-277b-4e44-9301-dd396b0a83d1-istiod-ca-cert\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.202615 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.202595 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-envoy\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.202812 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.202795 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-podinfo\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.208877 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.208843 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-token\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.209018 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.208998 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fzjp\" (UniqueName: \"kubernetes.io/projected/f55ee3da-277b-4e44-9301-dd396b0a83d1-kube-api-access-4fzjp\") pod \"e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.326457 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.326421 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:09.471280 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.471248 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:09.484873 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.484850 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd"] Jul 30 01:16:09.486566 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:16:09.486534 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf55ee3da_277b_4e44_9301_dd396b0a83d1.slice/crio-8cbba04281565a22856de207a0aff5774c6fa98abbdb7e1ec78cc2634153b701 WatchSource:0}: Error finding container 8cbba04281565a22856de207a0aff5774c6fa98abbdb7e1ec78cc2634153b701: Status 404 returned error can't find the container with id 8cbba04281565a22856de207a0aff5774c6fa98abbdb7e1ec78cc2634153b701 Jul 30 01:16:09.488910 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.488871 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:16:09.489004 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.488954 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:16:09.489004 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.488982 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:16:09.861424 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.861385 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" event={"ID":"f55ee3da-277b-4e44-9301-dd396b0a83d1","Type":"ContainerStarted","Data":"90255db4800819e22ec3b26721638abd08036e984be76c2786a502870c0c00ea"} Jul 30 01:16:09.861799 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.861447 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" event={"ID":"f55ee3da-277b-4e44-9301-dd396b0a83d1","Type":"ContainerStarted","Data":"8cbba04281565a22856de207a0aff5774c6fa98abbdb7e1ec78cc2634153b701"} Jul 30 01:16:09.884685 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:09.884613 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" podStartSLOduration=1.884593217 podStartE2EDuration="1.884593217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:16:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:16:09.882005474 +0000 UTC m=+1972.682962581" watchObservedRunningTime="2026-07-30 01:16:09.884593217 +0000 UTC m=+1972.685550325" Jul 30 01:16:10.327601 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:10.327562 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:10.332650 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:10.332620 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:10.868454 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:10.868415 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:10.869425 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:10.869403 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:11.356197 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:11.356154 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:11.501260 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:11.501216 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:11.836817 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:11.836779 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:11.976670 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:11.976621 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:12.076071 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:12.076039 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:13.296218 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.296194 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:16:13.337913 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.337869 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-workload-certs\") pod \"3a715981-3d30-469a-ba92-828fce07a2e4\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " Jul 30 01:16:13.338098 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.337939 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2sh6\" (UniqueName: \"kubernetes.io/projected/3a715981-3d30-469a-ba92-828fce07a2e4-kube-api-access-g2sh6\") pod \"3a715981-3d30-469a-ba92-828fce07a2e4\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " Jul 30 01:16:13.338098 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.337970 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a715981-3d30-469a-ba92-828fce07a2e4-istio-podinfo\") pod \"3a715981-3d30-469a-ba92-828fce07a2e4\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " Jul 30 01:16:13.338098 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.337994 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-istio-envoy\") pod \"3a715981-3d30-469a-ba92-828fce07a2e4\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " Jul 30 01:16:13.338098 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.338046 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-istio-data\") pod \"3a715981-3d30-469a-ba92-828fce07a2e4\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " Jul 30 01:16:13.338098 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.338072 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-workload-socket\") pod \"3a715981-3d30-469a-ba92-828fce07a2e4\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " Jul 30 01:16:13.338378 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.338104 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-credential-socket\") pod \"3a715981-3d30-469a-ba92-828fce07a2e4\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " Jul 30 01:16:13.338378 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.338150 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a715981-3d30-469a-ba92-828fce07a2e4-istio-token\") pod \"3a715981-3d30-469a-ba92-828fce07a2e4\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " Jul 30 01:16:13.338378 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.338179 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3a715981-3d30-469a-ba92-828fce07a2e4" (UID: "3a715981-3d30-469a-ba92-828fce07a2e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:16:13.338378 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.338194 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a715981-3d30-469a-ba92-828fce07a2e4-istiod-ca-cert\") pod \"3a715981-3d30-469a-ba92-828fce07a2e4\" (UID: \"3a715981-3d30-469a-ba92-828fce07a2e4\") " Jul 30 01:16:13.338585 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.338407 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3a715981-3d30-469a-ba92-828fce07a2e4" (UID: "3a715981-3d30-469a-ba92-828fce07a2e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:16:13.338648 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.338595 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-workload-certs\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:13.338648 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.338612 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-credential-socket\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:13.338758 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.338726 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a715981-3d30-469a-ba92-828fce07a2e4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3a715981-3d30-469a-ba92-828fce07a2e4" (UID: "3a715981-3d30-469a-ba92-828fce07a2e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:16:13.338812 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.338769 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3a715981-3d30-469a-ba92-828fce07a2e4" (UID: "3a715981-3d30-469a-ba92-828fce07a2e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:16:13.339276 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.339248 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-istio-data" (OuterVolumeSpecName: "istio-data") pod "3a715981-3d30-469a-ba92-828fce07a2e4" (UID: "3a715981-3d30-469a-ba92-828fce07a2e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:16:13.340965 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.340941 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a715981-3d30-469a-ba92-828fce07a2e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "3a715981-3d30-469a-ba92-828fce07a2e4" (UID: "3a715981-3d30-469a-ba92-828fce07a2e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:16:13.340965 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.340950 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3a715981-3d30-469a-ba92-828fce07a2e4" (UID: "3a715981-3d30-469a-ba92-828fce07a2e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:16:13.340965 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.340957 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a715981-3d30-469a-ba92-828fce07a2e4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3a715981-3d30-469a-ba92-828fce07a2e4" (UID: "3a715981-3d30-469a-ba92-828fce07a2e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 01:16:13.341154 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.340936 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a715981-3d30-469a-ba92-828fce07a2e4-kube-api-access-g2sh6" (OuterVolumeSpecName: "kube-api-access-g2sh6") pod "3a715981-3d30-469a-ba92-828fce07a2e4" (UID: "3a715981-3d30-469a-ba92-828fce07a2e4"). InnerVolumeSpecName "kube-api-access-g2sh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:16:13.439315 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.439281 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a715981-3d30-469a-ba92-828fce07a2e4-istio-token\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:13.439315 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.439310 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a715981-3d30-469a-ba92-828fce07a2e4-istiod-ca-cert\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:13.439315 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.439323 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2sh6\" (UniqueName: \"kubernetes.io/projected/3a715981-3d30-469a-ba92-828fce07a2e4-kube-api-access-g2sh6\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:13.439533 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.439332 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a715981-3d30-469a-ba92-828fce07a2e4-istio-podinfo\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:13.439533 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.439340 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-istio-envoy\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:13.439533 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.439348 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-istio-data\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:13.439533 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.439355 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a715981-3d30-469a-ba92-828fce07a2e4-workload-socket\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:13.884917 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.884870 2564 generic.go:356] "Generic (PLEG): container finished" podID="3a715981-3d30-469a-ba92-828fce07a2e4" containerID="1d4f87680c472346083a36056d5733b504fa1651881dfa3faae0ffdd6955c702" exitCode=0 Jul 30 01:16:13.885130 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.884937 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" event={"ID":"3a715981-3d30-469a-ba92-828fce07a2e4","Type":"ContainerDied","Data":"1d4f87680c472346083a36056d5733b504fa1651881dfa3faae0ffdd6955c702"} Jul 30 01:16:13.885130 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.884970 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" Jul 30 01:16:13.885130 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.884989 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" event={"ID":"3a715981-3d30-469a-ba92-828fce07a2e4","Type":"ContainerDied","Data":"b9668518871ff524c39466cb04d63b2daa0adb7454c3a7d280a75902028ae85f"} Jul 30 01:16:13.885130 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.885013 2564 scope.go:117] "RemoveContainer" containerID="1d4f87680c472346083a36056d5733b504fa1651881dfa3faae0ffdd6955c702" Jul 30 01:16:13.893978 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.893957 2564 scope.go:117] "RemoveContainer" containerID="ef5af0138a88fb022aec36cdf7405ca55dcc3481e73cfac6d05f12b594f4b816" Jul 30 01:16:13.902566 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.902541 2564 scope.go:117] "RemoveContainer" containerID="1d4f87680c472346083a36056d5733b504fa1651881dfa3faae0ffdd6955c702" Jul 30 01:16:13.902923 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:16:13.902883 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d4f87680c472346083a36056d5733b504fa1651881dfa3faae0ffdd6955c702\": container with ID starting with 1d4f87680c472346083a36056d5733b504fa1651881dfa3faae0ffdd6955c702 not found: ID does not exist" containerID="1d4f87680c472346083a36056d5733b504fa1651881dfa3faae0ffdd6955c702" Jul 30 01:16:13.902975 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.902935 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d4f87680c472346083a36056d5733b504fa1651881dfa3faae0ffdd6955c702"} err="failed to get container status \"1d4f87680c472346083a36056d5733b504fa1651881dfa3faae0ffdd6955c702\": rpc error: code = NotFound desc = could not find container \"1d4f87680c472346083a36056d5733b504fa1651881dfa3faae0ffdd6955c702\": container with ID starting with 1d4f87680c472346083a36056d5733b504fa1651881dfa3faae0ffdd6955c702 not found: ID does not exist" Jul 30 01:16:13.902975 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.902954 2564 scope.go:117] "RemoveContainer" containerID="ef5af0138a88fb022aec36cdf7405ca55dcc3481e73cfac6d05f12b594f4b816" Jul 30 01:16:13.903226 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:16:13.903171 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef5af0138a88fb022aec36cdf7405ca55dcc3481e73cfac6d05f12b594f4b816\": container with ID starting with ef5af0138a88fb022aec36cdf7405ca55dcc3481e73cfac6d05f12b594f4b816 not found: ID does not exist" containerID="ef5af0138a88fb022aec36cdf7405ca55dcc3481e73cfac6d05f12b594f4b816" Jul 30 01:16:13.903226 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.903188 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef5af0138a88fb022aec36cdf7405ca55dcc3481e73cfac6d05f12b594f4b816"} err="failed to get container status \"ef5af0138a88fb022aec36cdf7405ca55dcc3481e73cfac6d05f12b594f4b816\": rpc error: code = NotFound desc = could not find container \"ef5af0138a88fb022aec36cdf7405ca55dcc3481e73cfac6d05f12b594f4b816\": container with ID starting with ef5af0138a88fb022aec36cdf7405ca55dcc3481e73cfac6d05f12b594f4b816 not found: ID does not exist" Jul 30 01:16:13.906495 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.906469 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k"] Jul 30 01:16:13.910330 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:13.910305 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k"] Jul 30 01:16:14.283149 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:14.283103 2564 patch_prober.go:28] interesting pod/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 30 01:16:14.283312 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:14.283172 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-76aaf06c-openshift-default-594dd56856-fh72k" podUID="3a715981-3d30-469a-ba92-828fce07a2e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 30 01:16:15.669502 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:15.669466 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a715981-3d30-469a-ba92-828fce07a2e4" path="/var/lib/kubelet/pods/3a715981-3d30-469a-ba92-828fce07a2e4/volumes" Jul 30 01:16:26.177775 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:26.176929 2564 patch_prober.go:28] interesting pod/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:45990->10.132.0.42:15021: read: connection reset by peer" start-of-body= Jul 30 01:16:26.177775 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:26.176996 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" podUID="f55ee3da-277b-4e44-9301-dd396b0a83d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:45990->10.132.0.42:15021: read: connection reset by peer" Jul 30 01:16:26.964370 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:26.964336 2564 generic.go:356] "Generic (PLEG): container finished" podID="f55ee3da-277b-4e44-9301-dd396b0a83d1" containerID="90255db4800819e22ec3b26721638abd08036e984be76c2786a502870c0c00ea" exitCode=0 Jul 30 01:16:26.964549 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:26.964407 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" event={"ID":"f55ee3da-277b-4e44-9301-dd396b0a83d1","Type":"ContainerDied","Data":"90255db4800819e22ec3b26721638abd08036e984be76c2786a502870c0c00ea"} Jul 30 01:16:26.964822 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:26.964808 2564 scope.go:117] "RemoveContainer" containerID="90255db4800819e22ec3b26721638abd08036e984be76c2786a502870c0c00ea" Jul 30 01:16:26.965840 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:26.965815 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:16:26.965888 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:26.965881 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:16:26.965953 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:26.965927 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:16:27.972470 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:27.972436 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" event={"ID":"f55ee3da-277b-4e44-9301-dd396b0a83d1","Type":"ContainerStarted","Data":"ee4c4dcea3afa3d8083d5e37f64dadae84b9f1bb36f06f98cb9175991ab0e5b6"} Jul 30 01:16:28.327019 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:28.326979 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:28.327355 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:28.327332 2564 patch_prober.go:28] interesting pod/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 30 01:16:28.327420 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:28.327400 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" podUID="f55ee3da-277b-4e44-9301-dd396b0a83d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 30 01:16:29.327721 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:29.327679 2564 patch_prober.go:28] interesting pod/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 30 01:16:29.328173 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:29.327748 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" podUID="f55ee3da-277b-4e44-9301-dd396b0a83d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 30 01:16:30.327290 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:30.327254 2564 patch_prober.go:28] interesting pod/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 30 01:16:30.327482 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:30.327311 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" podUID="f55ee3da-277b-4e44-9301-dd396b0a83d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 30 01:16:31.327085 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:31.327040 2564 patch_prober.go:28] interesting pod/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 30 01:16:31.327486 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:31.327103 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" podUID="f55ee3da-277b-4e44-9301-dd396b0a83d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 30 01:16:32.330596 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:32.330564 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:32.331035 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:32.330931 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:32.331685 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:32.331669 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:35.643368 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:35.643264 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:35.743945 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:35.743883 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:42.825916 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:42.825860 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:42.925829 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:42.925793 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:43.040159 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:43.040124 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:43.117799 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:43.117718 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd"] Jul 30 01:16:43.118083 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:43.118055 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" podUID="f55ee3da-277b-4e44-9301-dd396b0a83d1" containerName="istio-proxy" containerID="cri-o://ee4c4dcea3afa3d8083d5e37f64dadae84b9f1bb36f06f98cb9175991ab0e5b6" gracePeriod=30 Jul 30 01:16:43.143802 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:43.143773 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:43.418749 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:43.418646 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:43.601697 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:43.601661 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:43.914783 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:43.914748 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:44.326274 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:44.326230 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:47.333942 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:47.333889 2564 patch_prober.go:28] interesting pod/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 01:16:47.334349 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:47.333964 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" podUID="f55ee3da-277b-4e44-9301-dd396b0a83d1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 01:16:48.380043 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.379979 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:48.396484 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.396441 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fzjp\" (UniqueName: \"kubernetes.io/projected/f55ee3da-277b-4e44-9301-dd396b0a83d1-kube-api-access-4fzjp\") pod \"f55ee3da-277b-4e44-9301-dd396b0a83d1\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " Jul 30 01:16:48.396656 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.396513 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-podinfo\") pod \"f55ee3da-277b-4e44-9301-dd396b0a83d1\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " Jul 30 01:16:48.396656 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.396539 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-envoy\") pod \"f55ee3da-277b-4e44-9301-dd396b0a83d1\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " Jul 30 01:16:48.396656 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.396567 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-data\") pod \"f55ee3da-277b-4e44-9301-dd396b0a83d1\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " Jul 30 01:16:48.396656 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.396592 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-token\") pod \"f55ee3da-277b-4e44-9301-dd396b0a83d1\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " Jul 30 01:16:48.396656 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.396627 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-workload-socket\") pod \"f55ee3da-277b-4e44-9301-dd396b0a83d1\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " Jul 30 01:16:48.396656 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.396651 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-workload-certs\") pod \"f55ee3da-277b-4e44-9301-dd396b0a83d1\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " Jul 30 01:16:48.397003 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.396673 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-credential-socket\") pod \"f55ee3da-277b-4e44-9301-dd396b0a83d1\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " Jul 30 01:16:48.397003 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.396701 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f55ee3da-277b-4e44-9301-dd396b0a83d1-istiod-ca-cert\") pod \"f55ee3da-277b-4e44-9301-dd396b0a83d1\" (UID: \"f55ee3da-277b-4e44-9301-dd396b0a83d1\") " Jul 30 01:16:48.397120 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.397057 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f55ee3da-277b-4e44-9301-dd396b0a83d1" (UID: "f55ee3da-277b-4e44-9301-dd396b0a83d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:16:48.397312 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.397268 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f55ee3da-277b-4e44-9301-dd396b0a83d1" (UID: "f55ee3da-277b-4e44-9301-dd396b0a83d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:16:48.397391 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.397302 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f55ee3da-277b-4e44-9301-dd396b0a83d1" (UID: "f55ee3da-277b-4e44-9301-dd396b0a83d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:16:48.397391 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.397310 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f55ee3da-277b-4e44-9301-dd396b0a83d1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f55ee3da-277b-4e44-9301-dd396b0a83d1" (UID: "f55ee3da-277b-4e44-9301-dd396b0a83d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:16:48.397506 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.397445 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-data" (OuterVolumeSpecName: "istio-data") pod "f55ee3da-277b-4e44-9301-dd396b0a83d1" (UID: "f55ee3da-277b-4e44-9301-dd396b0a83d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:16:48.398821 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.398800 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f55ee3da-277b-4e44-9301-dd396b0a83d1" (UID: "f55ee3da-277b-4e44-9301-dd396b0a83d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 01:16:48.398989 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.398967 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f55ee3da-277b-4e44-9301-dd396b0a83d1" (UID: "f55ee3da-277b-4e44-9301-dd396b0a83d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:16:48.399450 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.399424 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-token" (OuterVolumeSpecName: "istio-token") pod "f55ee3da-277b-4e44-9301-dd396b0a83d1" (UID: "f55ee3da-277b-4e44-9301-dd396b0a83d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:16:48.399511 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.399459 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f55ee3da-277b-4e44-9301-dd396b0a83d1-kube-api-access-4fzjp" (OuterVolumeSpecName: "kube-api-access-4fzjp") pod "f55ee3da-277b-4e44-9301-dd396b0a83d1" (UID: "f55ee3da-277b-4e44-9301-dd396b0a83d1"). InnerVolumeSpecName "kube-api-access-4fzjp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:16:48.496404 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.496359 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:48.497647 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.497628 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-podinfo\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:48.497705 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.497650 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-envoy\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:48.497705 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.497660 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-data\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:48.497705 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.497668 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f55ee3da-277b-4e44-9301-dd396b0a83d1-istio-token\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:48.497705 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.497676 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-workload-socket\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:48.497705 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.497684 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-workload-certs\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:48.497705 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.497693 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f55ee3da-277b-4e44-9301-dd396b0a83d1-credential-socket\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:48.497705 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.497701 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f55ee3da-277b-4e44-9301-dd396b0a83d1-istiod-ca-cert\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:48.497936 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.497709 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fzjp\" (UniqueName: \"kubernetes.io/projected/f55ee3da-277b-4e44-9301-dd396b0a83d1-kube-api-access-4fzjp\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:16:48.596174 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:48.596130 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:49.089097 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:49.089063 2564 generic.go:356] "Generic (PLEG): container finished" podID="f55ee3da-277b-4e44-9301-dd396b0a83d1" containerID="ee4c4dcea3afa3d8083d5e37f64dadae84b9f1bb36f06f98cb9175991ab0e5b6" exitCode=0 Jul 30 01:16:49.089276 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:49.089126 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" Jul 30 01:16:49.089276 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:49.089149 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" event={"ID":"f55ee3da-277b-4e44-9301-dd396b0a83d1","Type":"ContainerDied","Data":"ee4c4dcea3afa3d8083d5e37f64dadae84b9f1bb36f06f98cb9175991ab0e5b6"} Jul 30 01:16:49.089276 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:49.089208 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd" event={"ID":"f55ee3da-277b-4e44-9301-dd396b0a83d1","Type":"ContainerDied","Data":"8cbba04281565a22856de207a0aff5774c6fa98abbdb7e1ec78cc2634153b701"} Jul 30 01:16:49.089276 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:49.089240 2564 scope.go:117] "RemoveContainer" containerID="ee4c4dcea3afa3d8083d5e37f64dadae84b9f1bb36f06f98cb9175991ab0e5b6" Jul 30 01:16:49.099266 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:49.099243 2564 scope.go:117] "RemoveContainer" containerID="90255db4800819e22ec3b26721638abd08036e984be76c2786a502870c0c00ea" Jul 30 01:16:49.107786 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:49.107768 2564 scope.go:117] "RemoveContainer" containerID="ee4c4dcea3afa3d8083d5e37f64dadae84b9f1bb36f06f98cb9175991ab0e5b6" Jul 30 01:16:49.108091 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:16:49.108073 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee4c4dcea3afa3d8083d5e37f64dadae84b9f1bb36f06f98cb9175991ab0e5b6\": container with ID starting with ee4c4dcea3afa3d8083d5e37f64dadae84b9f1bb36f06f98cb9175991ab0e5b6 not found: ID does not exist" containerID="ee4c4dcea3afa3d8083d5e37f64dadae84b9f1bb36f06f98cb9175991ab0e5b6" Jul 30 01:16:49.108149 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:49.108099 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee4c4dcea3afa3d8083d5e37f64dadae84b9f1bb36f06f98cb9175991ab0e5b6"} err="failed to get container status \"ee4c4dcea3afa3d8083d5e37f64dadae84b9f1bb36f06f98cb9175991ab0e5b6\": rpc error: code = NotFound desc = could not find container \"ee4c4dcea3afa3d8083d5e37f64dadae84b9f1bb36f06f98cb9175991ab0e5b6\": container with ID starting with ee4c4dcea3afa3d8083d5e37f64dadae84b9f1bb36f06f98cb9175991ab0e5b6 not found: ID does not exist" Jul 30 01:16:49.108149 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:49.108116 2564 scope.go:117] "RemoveContainer" containerID="90255db4800819e22ec3b26721638abd08036e984be76c2786a502870c0c00ea" Jul 30 01:16:49.108360 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:16:49.108341 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90255db4800819e22ec3b26721638abd08036e984be76c2786a502870c0c00ea\": container with ID starting with 90255db4800819e22ec3b26721638abd08036e984be76c2786a502870c0c00ea not found: ID does not exist" containerID="90255db4800819e22ec3b26721638abd08036e984be76c2786a502870c0c00ea" Jul 30 01:16:49.108403 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:49.108367 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90255db4800819e22ec3b26721638abd08036e984be76c2786a502870c0c00ea"} err="failed to get container status \"90255db4800819e22ec3b26721638abd08036e984be76c2786a502870c0c00ea\": rpc error: code = NotFound desc = could not find container \"90255db4800819e22ec3b26721638abd08036e984be76c2786a502870c0c00ea\": container with ID starting with 90255db4800819e22ec3b26721638abd08036e984be76c2786a502870c0c00ea not found: ID does not exist" Jul 30 01:16:49.113191 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:49.113171 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd"] Jul 30 01:16:49.117690 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:49.117668 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-a6605e44-gw-openshift-default-5f7cb8b584-2kksd"] Jul 30 01:16:49.668858 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:49.668821 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f55ee3da-277b-4e44-9301-dd396b0a83d1" path="/var/lib/kubelet/pods/f55ee3da-277b-4e44-9301-dd396b0a83d1/volumes" Jul 30 01:16:49.795571 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:49.795540 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:49.952555 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:49.952467 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:50.332707 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:50.332676 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:16:50.432581 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:16:50.432548 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:20.318369 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:20.318334 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:20.382657 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:20.382618 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:20.441953 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:20.441862 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:20.506584 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:20.506532 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:21.622459 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:21.622424 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:21.832154 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:21.832121 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:22.053961 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:22.053847 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:22.359098 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:22.359067 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:22.514327 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:22.514288 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:24.682298 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:24.682258 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:24.781032 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:24.780997 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:25.817854 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:25.817811 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:26.312477 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:26.312381 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:26.411990 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:26.411961 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:56.496574 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:56.496488 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:56.599509 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:56.599451 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:56.683179 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:56.683146 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:56.741043 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:56.740997 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:56.828540 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:56.828512 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:57.046780 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:57.046738 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:57.511076 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:57.511044 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:57.609118 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:57.609024 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:17:57.677173 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:17:57.677139 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:18:13.152857 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:18:13.152819 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:18:19.561861 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:18:19.561822 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:18:25.368664 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:18:25.368624 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:18:25.465347 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:18:25.465299 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:18:25.679715 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:18:25.679592 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:18:25.776284 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:18:25.776252 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:18:26.044674 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:18:26.044563 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:18:51.958464 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:18:51.958418 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:18:58.252754 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:18:58.252714 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:18:58.358374 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:18:58.358333 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:18:58.453301 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:18:58.453253 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:14.337333 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:14.337231 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:19.923125 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:19.923094 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:19.983083 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:19.983043 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:20.065582 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:20.065535 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:20.168244 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:20.168207 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:30.637425 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:30.637385 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:36.253986 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:36.253950 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:36.310853 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:36.310826 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:41.657742 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:41.657704 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:47.930325 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:47.930287 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:48.038051 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:48.038012 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:48.131173 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:48.131140 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:48.230491 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:48.230407 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:48.871420 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:48.871385 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:48.962948 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:48.962885 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:49.070554 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:49.070496 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:50.535803 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:50.535770 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:56.341294 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:56.341259 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:56.416181 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:56.416147 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:19:57.027148 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:19:57.027098 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:02.888013 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:02.887977 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:02.960168 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:02.960103 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:03.151781 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:03.151686 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:03.462416 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:03.462316 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:04.132612 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:04.132568 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:06.299484 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:06.299446 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:06.398317 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:06.398276 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:07.474560 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:07.474526 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:07.582437 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:07.582385 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:07.930944 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:07.930886 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:08.032015 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:08.031964 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:09.784909 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:09.784859 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:14.848400 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:14.848354 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:31.219830 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:31.219790 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:33.972696 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:33.972662 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:38.264972 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:38.264928 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:38.356718 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:38.356684 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:49.090556 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:49.090479 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:49.155876 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:49.155842 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:49.215034 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:49.215002 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:49.272786 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:49.272719 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:49.625072 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:49.625037 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:55.440859 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:55.440808 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:55.541403 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:55.541373 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:55.688668 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:55.688630 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:55.895258 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:55.895227 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:56.020520 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:56.020462 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:56.118669 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:56.118634 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:20:56.270864 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:20:56.270783 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:21:02.052760 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:21:02.052728 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:21:02.154556 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:21:02.154515 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:21:02.252991 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:21:02.252953 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:21:02.398133 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:21:02.398076 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:21:08.232921 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:21:08.232864 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:21:08.294817 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:21:08.294779 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:23:10.621104 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:23:10.621069 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:23:10.883381 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:23:10.883284 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:23:11.431532 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:23:11.431490 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:23:13.952506 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:23:13.952468 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:23:14.052165 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:23:14.052133 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:23:15.081636 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:23:15.081583 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:23:15.706342 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:23:15.706312 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:23:15.803305 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:23:15.803266 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:23:38.959581 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:23:38.959540 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:23:52.571614 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:23:52.571575 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:24:39.024185 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:24:39.024150 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:24:39.090815 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:24:39.090779 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:24:39.154887 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:24:39.154841 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:24:39.210066 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:24:39.209877 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:24:42.321189 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:24:42.321153 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:24:47.399084 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:24:47.399041 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:24:51.502257 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:24:51.502221 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:25:24.264327 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:25:24.264283 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:25:24.452015 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:25:24.451973 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:25:30.038771 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:25:30.038733 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:25:44.270936 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:25:44.270871 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:25:45.841634 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:25:45.841596 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:25:56.148676 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:25:56.148632 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:25:56.247524 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:25:56.247491 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:25:56.347197 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:25:56.347153 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:25:57.033457 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:25:57.033424 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:25:57.131702 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:25:57.131657 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:25:58.481499 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:25:58.481465 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:04.397201 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:04.397156 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:04.505812 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:04.505775 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:05.081332 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:05.081294 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:10.697988 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:10.697952 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:10.799687 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:10.799655 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:10.898025 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:10.897986 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:14.096012 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:14.095969 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:21.416583 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:21.416550 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:21.652616 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:21.652576 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:42.584145 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:42.584049 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:48.624521 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:48.624478 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:48.714600 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:48.714562 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:48.972124 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:48.972027 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:49.282047 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:49.281953 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:49.644278 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:49.644246 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:52.003209 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:52.003169 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:52.104749 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:52.104710 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:53.280756 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:53.280700 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:53.723411 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:53.723357 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:53.825728 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:53.825698 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:26:53.892737 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:26:53.892694 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:27:17.028930 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.028871 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:27:17.366866 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.366832 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:27:17.676883 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.676808 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt"] Jul 30 01:27:17.677265 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.677250 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f55ee3da-277b-4e44-9301-dd396b0a83d1" containerName="istio-proxy" Jul 30 01:27:17.677318 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.677266 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f55ee3da-277b-4e44-9301-dd396b0a83d1" containerName="istio-proxy" Jul 30 01:27:17.677318 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.677274 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f55ee3da-277b-4e44-9301-dd396b0a83d1" containerName="istio-proxy" Jul 30 01:27:17.677318 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.677280 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f55ee3da-277b-4e44-9301-dd396b0a83d1" containerName="istio-proxy" Jul 30 01:27:17.677318 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.677290 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a715981-3d30-469a-ba92-828fce07a2e4" containerName="istio-proxy" Jul 30 01:27:17.677318 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.677295 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a715981-3d30-469a-ba92-828fce07a2e4" containerName="istio-proxy" Jul 30 01:27:17.677318 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.677302 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a715981-3d30-469a-ba92-828fce07a2e4" containerName="istio-proxy" Jul 30 01:27:17.677318 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.677308 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a715981-3d30-469a-ba92-828fce07a2e4" containerName="istio-proxy" Jul 30 01:27:17.677542 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.677389 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f55ee3da-277b-4e44-9301-dd396b0a83d1" containerName="istio-proxy" Jul 30 01:27:17.677542 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.677397 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f55ee3da-277b-4e44-9301-dd396b0a83d1" containerName="istio-proxy" Jul 30 01:27:17.677542 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.677404 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a715981-3d30-469a-ba92-828fce07a2e4" containerName="istio-proxy" Jul 30 01:27:17.677542 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.677410 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a715981-3d30-469a-ba92-828fce07a2e4" containerName="istio-proxy" Jul 30 01:27:17.680414 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.680395 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.682759 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.682736 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-b2ddd3-openshift-default-dockercfg-8dl9l\"" Jul 30 01:27:17.690658 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.690630 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt"] Jul 30 01:27:17.704956 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.704923 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:27:17.801384 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.801351 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-data\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.801582 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.801487 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-podinfo\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.801582 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.801529 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-workload-certs\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.801582 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.801549 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-token\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.801765 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.801632 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-envoy\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.801765 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.801662 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-credential-socket\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.801765 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.801705 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istiod-ca-cert\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.801765 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.801748 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7mpg\" (UniqueName: \"kubernetes.io/projected/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-kube-api-access-n7mpg\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.801948 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.801773 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-workload-socket\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.896931 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.896880 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:27:17.902297 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.902270 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-envoy\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.902406 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.902306 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-credential-socket\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.902406 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.902344 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istiod-ca-cert\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.902406 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.902371 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7mpg\" (UniqueName: \"kubernetes.io/projected/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-kube-api-access-n7mpg\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.902406 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.902390 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-workload-socket\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.902633 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.902422 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-data\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.902633 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.902513 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-podinfo\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.902633 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.902548 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-workload-certs\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.902633 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.902571 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-token\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.902849 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.902779 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-credential-socket\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.902972 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.902954 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-workload-socket\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.903213 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.903191 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istiod-ca-cert\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.903303 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.903260 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-workload-certs\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.903303 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.903285 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-data\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.904834 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.904814 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-envoy\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.905474 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.905448 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-podinfo\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.910761 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.910735 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7mpg\" (UniqueName: \"kubernetes.io/projected/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-kube-api-access-n7mpg\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.911085 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.911061 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-token\") pod \"e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:17.991565 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:17.991472 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:18.132599 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:18.132573 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt"] Jul 30 01:27:18.134280 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:27:18.134254 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf644aece_e0e8_4abd_83ed_ca7f5ddca5fc.slice/crio-710b5560f8f8ecf4601fca0744cce4d14b786062c9b9a16b7207b49d6afe405e WatchSource:0}: Error finding container 710b5560f8f8ecf4601fca0744cce4d14b786062c9b9a16b7207b49d6afe405e: Status 404 returned error can't find the container with id 710b5560f8f8ecf4601fca0744cce4d14b786062c9b9a16b7207b49d6afe405e Jul 30 01:27:18.136967 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:18.136924 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:27:18.137060 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:18.137006 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:27:18.137060 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:18.137052 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:27:18.468722 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:18.468684 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" event={"ID":"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc","Type":"ContainerStarted","Data":"08c68ca4476c0c82cda00c6fead6d8954d6f1042cb9b96285126f7f58c8b9af8"} Jul 30 01:27:18.468914 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:18.468743 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" event={"ID":"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc","Type":"ContainerStarted","Data":"710b5560f8f8ecf4601fca0744cce4d14b786062c9b9a16b7207b49d6afe405e"} Jul 30 01:27:18.488651 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:18.488601 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" podStartSLOduration=1.488585503 podStartE2EDuration="1.488585503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:27:18.486290917 +0000 UTC m=+2641.287248024" watchObservedRunningTime="2026-07-30 01:27:18.488585503 +0000 UTC m=+2641.289542673" Jul 30 01:27:18.992367 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:18.992329 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:19.997273 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:19.997244 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:20.381472 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:20.381440 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:27:20.480647 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:20.480619 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:20.481639 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:20.481618 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:20.483379 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:20.483353 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:27:21.504815 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:21.504776 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:27:21.717824 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:21.717774 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:27:22.054633 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:22.054597 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:27:22.154034 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:22.154000 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:27:36.432225 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:36.430831 2564 patch_prober.go:28] interesting pod/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": read tcp 10.132.0.2:53754->10.132.0.43:15021: read: connection reset by peer" start-of-body= Jul 30 01:27:36.432225 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:36.430893 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" podUID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": read tcp 10.132.0.2:53754->10.132.0.43:15021: read: connection reset by peer" Jul 30 01:27:36.568711 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:36.568673 2564 generic.go:356] "Generic (PLEG): container finished" podID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" containerID="08c68ca4476c0c82cda00c6fead6d8954d6f1042cb9b96285126f7f58c8b9af8" exitCode=0 Jul 30 01:27:36.568890 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:36.568748 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" event={"ID":"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc","Type":"ContainerDied","Data":"08c68ca4476c0c82cda00c6fead6d8954d6f1042cb9b96285126f7f58c8b9af8"} Jul 30 01:27:36.569242 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:36.569227 2564 scope.go:117] "RemoveContainer" containerID="08c68ca4476c0c82cda00c6fead6d8954d6f1042cb9b96285126f7f58c8b9af8" Jul 30 01:27:36.570379 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:36.570351 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:27:36.570434 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:36.570419 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:27:36.570470 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:36.570448 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:27:37.575413 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:37.575380 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" event={"ID":"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc","Type":"ContainerStarted","Data":"7528863bf70b705a0ea703c89aa5d51c7e0b987caa7c6ad9aa837375577289ce"} Jul 30 01:27:37.992777 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:37.992675 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:37.992966 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:37.992948 2564 patch_prober.go:28] interesting pod/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 30 01:27:37.993031 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:37.993011 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" podUID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 30 01:27:38.992723 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:38.992684 2564 patch_prober.go:28] interesting pod/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 30 01:27:38.993151 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:38.992744 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" podUID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 30 01:27:39.992707 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:39.992669 2564 patch_prober.go:28] interesting pod/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 30 01:27:39.992923 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:39.992726 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" podUID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 30 01:27:40.992162 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:40.992126 2564 patch_prober.go:28] interesting pod/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 30 01:27:40.992346 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:40.992194 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" podUID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 30 01:27:41.995534 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:41.995502 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:41.996006 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:41.995957 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:41.996597 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:41.996580 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:27:45.391355 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:45.391316 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:27:49.143310 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:27:49.143265 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:28:39.545083 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:28:39.545044 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:28:48.417844 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:28:48.417808 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:28:48.477646 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:28:48.477606 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:20.354184 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:20.354139 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:21.353721 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:21.353679 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:21.450254 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:21.450211 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6ce027-zw522"] Jul 30 01:29:21.454976 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:21.454941 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6ce027-zw522" Jul 30 01:29:21.459563 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:21.459531 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\"" Jul 30 01:29:21.459889 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:21.459552 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 30 01:29:21.463326 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:21.463292 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6ce027-zw522"] Jul 30 01:29:21.506069 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:21.506026 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e6c7714-60fb-4cfb-b210-8131be6fc03a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6ce027-zw522\" (UID: \"7e6c7714-60fb-4cfb-b210-8131be6fc03a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6ce027-zw522" Jul 30 01:29:21.606829 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:21.606718 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e6c7714-60fb-4cfb-b210-8131be6fc03a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6ce027-zw522\" (UID: \"7e6c7714-60fb-4cfb-b210-8131be6fc03a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6ce027-zw522" Jul 30 01:29:21.607407 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:21.607384 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e6c7714-60fb-4cfb-b210-8131be6fc03a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6ce027-zw522\" (UID: \"7e6c7714-60fb-4cfb-b210-8131be6fc03a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6ce027-zw522" Jul 30 01:29:21.771508 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:21.771459 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6ce027-zw522" Jul 30 01:29:21.913050 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:21.913020 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6ce027-zw522"] Jul 30 01:29:21.915351 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:29:21.915321 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e6c7714_60fb_4cfb_b210_8131be6fc03a.slice/crio-f2f6dfde53ff525e19d5c945bf21eaeead2d5281c4ee1cae0a6573d0ef3eda42 WatchSource:0}: Error finding container f2f6dfde53ff525e19d5c945bf21eaeead2d5281c4ee1cae0a6573d0ef3eda42: Status 404 returned error can't find the container with id f2f6dfde53ff525e19d5c945bf21eaeead2d5281c4ee1cae0a6573d0ef3eda42 Jul 30 01:29:21.917469 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:21.917451 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 01:29:22.141922 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:22.141816 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6ce027-zw522" event={"ID":"7e6c7714-60fb-4cfb-b210-8131be6fc03a","Type":"ContainerStarted","Data":"f2f6dfde53ff525e19d5c945bf21eaeead2d5281c4ee1cae0a6573d0ef3eda42"} Jul 30 01:29:25.161785 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:25.161695 2564 generic.go:356] "Generic (PLEG): container finished" podID="7e6c7714-60fb-4cfb-b210-8131be6fc03a" containerID="dd353b795f470c4862c09e948218db44e4ae340cc8dc16620b10460cb39278af" exitCode=0 Jul 30 01:29:25.161785 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:25.161746 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6ce027-zw522" event={"ID":"7e6c7714-60fb-4cfb-b210-8131be6fc03a","Type":"ContainerDied","Data":"dd353b795f470c4862c09e948218db44e4ae340cc8dc16620b10460cb39278af"} Jul 30 01:29:26.304572 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:26.304540 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6ce027-zw522" Jul 30 01:29:26.354051 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:26.354013 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e6c7714-60fb-4cfb-b210-8131be6fc03a-maas-api-service-ca\") pod \"7e6c7714-60fb-4cfb-b210-8131be6fc03a\" (UID: \"7e6c7714-60fb-4cfb-b210-8131be6fc03a\") " Jul 30 01:29:26.354389 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:26.354366 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e6c7714-60fb-4cfb-b210-8131be6fc03a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7e6c7714-60fb-4cfb-b210-8131be6fc03a" (UID: "7e6c7714-60fb-4cfb-b210-8131be6fc03a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:29:26.455744 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:26.455653 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e6c7714-60fb-4cfb-b210-8131be6fc03a-maas-api-service-ca\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:29:27.175349 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:27.175311 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6ce027-zw522" Jul 30 01:29:27.175523 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:27.175348 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6ce027-zw522" event={"ID":"7e6c7714-60fb-4cfb-b210-8131be6fc03a","Type":"ContainerDied","Data":"f2f6dfde53ff525e19d5c945bf21eaeead2d5281c4ee1cae0a6573d0ef3eda42"} Jul 30 01:29:27.175523 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:27.175397 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2f6dfde53ff525e19d5c945bf21eaeead2d5281c4ee1cae0a6573d0ef3eda42" Jul 30 01:29:31.996670 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:31.996632 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:32.065042 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:32.064997 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:32.139817 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:32.139766 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:32.191956 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:32.191925 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6ce027-zw522"] Jul 30 01:29:32.204882 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:32.204834 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6ce027-zw522"] Jul 30 01:29:32.209401 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:32.209216 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:32.568889 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:32.568856 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:33.668905 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:33.668861 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e6c7714-60fb-4cfb-b210-8131be6fc03a" path="/var/lib/kubelet/pods/7e6c7714-60fb-4cfb-b210-8131be6fc03a/volumes" Jul 30 01:29:38.364978 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:38.364937 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:38.464278 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:38.464243 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:38.640809 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:38.640720 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:38.884061 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:38.884024 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:49.427819 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:49.427773 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:49.528545 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:49.528513 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:49.613939 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:49.613888 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:49.674862 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:49.674824 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:49.956846 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:49.956805 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:55.773449 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:55.773413 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:55.873686 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:55.873650 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:56.014495 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:56.014461 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt"] Jul 30 01:29:56.014725 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:56.014697 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" podUID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" containerName="istio-proxy" containerID="cri-o://7528863bf70b705a0ea703c89aa5d51c7e0b987caa7c6ad9aa837375577289ce" gracePeriod=30 Jul 30 01:29:56.035583 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:56.035480 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:56.262961 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:56.262925 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:56.604561 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:56.604527 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:56.931341 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:56.931254 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:56.998887 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:56.998850 2564 patch_prober.go:28] interesting pod/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 01:29:56.999066 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:56.998946 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" podUID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 01:29:57.027391 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:57.027355 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:59.842970 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:59.842930 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:29:59.943737 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:29:59.943697 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:30:00.985501 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:00.985454 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:30:01.286203 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.286175 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:30:01.375877 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.375845 2564 generic.go:356] "Generic (PLEG): container finished" podID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" containerID="7528863bf70b705a0ea703c89aa5d51c7e0b987caa7c6ad9aa837375577289ce" exitCode=0 Jul 30 01:30:01.376042 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.375913 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" Jul 30 01:30:01.376042 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.375962 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" event={"ID":"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc","Type":"ContainerDied","Data":"7528863bf70b705a0ea703c89aa5d51c7e0b987caa7c6ad9aa837375577289ce"} Jul 30 01:30:01.376042 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.376012 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt" event={"ID":"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc","Type":"ContainerDied","Data":"710b5560f8f8ecf4601fca0744cce4d14b786062c9b9a16b7207b49d6afe405e"} Jul 30 01:30:01.376042 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.376036 2564 scope.go:117] "RemoveContainer" containerID="7528863bf70b705a0ea703c89aa5d51c7e0b987caa7c6ad9aa837375577289ce" Jul 30 01:30:01.385302 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.385280 2564 scope.go:117] "RemoveContainer" containerID="08c68ca4476c0c82cda00c6fead6d8954d6f1042cb9b96285126f7f58c8b9af8" Jul 30 01:30:01.392753 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.392722 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-envoy\") pod \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " Jul 30 01:30:01.392880 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.392772 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-token\") pod \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " Jul 30 01:30:01.392880 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.392811 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-workload-certs\") pod \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " Jul 30 01:30:01.392880 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.392839 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-workload-socket\") pod \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " Jul 30 01:30:01.393023 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.392892 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-podinfo\") pod \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " Jul 30 01:30:01.393023 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.392960 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istiod-ca-cert\") pod \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " Jul 30 01:30:01.393023 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.392988 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-credential-socket\") pod \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " Jul 30 01:30:01.393193 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.393029 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-data\") pod \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " Jul 30 01:30:01.393193 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.393055 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7mpg\" (UniqueName: \"kubernetes.io/projected/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-kube-api-access-n7mpg\") pod \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\" (UID: \"f644aece-e0e8-4abd-83ed-ca7f5ddca5fc\") " Jul 30 01:30:01.393305 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.393248 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" (UID: "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:30:01.393305 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.393252 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" (UID: "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:30:01.393436 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.393409 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-workload-certs\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:30:01.393499 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.393434 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-workload-socket\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:30:01.393656 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.393617 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" (UID: "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:30:01.393656 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.393628 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" (UID: "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:30:01.394241 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.394213 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" (UID: "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:30:01.395263 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.395235 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" (UID: "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:30:01.395354 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.395280 2564 scope.go:117] "RemoveContainer" containerID="7528863bf70b705a0ea703c89aa5d51c7e0b987caa7c6ad9aa837375577289ce" Jul 30 01:30:01.395499 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.395476 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" (UID: "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:30:01.395645 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.395626 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-kube-api-access-n7mpg" (OuterVolumeSpecName: "kube-api-access-n7mpg") pod "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" (UID: "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc"). InnerVolumeSpecName "kube-api-access-n7mpg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:30:01.395703 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.395645 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" (UID: "f644aece-e0e8-4abd-83ed-ca7f5ddca5fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 01:30:01.395703 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:30:01.395644 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7528863bf70b705a0ea703c89aa5d51c7e0b987caa7c6ad9aa837375577289ce\": container with ID starting with 7528863bf70b705a0ea703c89aa5d51c7e0b987caa7c6ad9aa837375577289ce not found: ID does not exist" containerID="7528863bf70b705a0ea703c89aa5d51c7e0b987caa7c6ad9aa837375577289ce" Jul 30 01:30:01.395703 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.395678 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7528863bf70b705a0ea703c89aa5d51c7e0b987caa7c6ad9aa837375577289ce"} err="failed to get container status \"7528863bf70b705a0ea703c89aa5d51c7e0b987caa7c6ad9aa837375577289ce\": rpc error: code = NotFound desc = could not find container \"7528863bf70b705a0ea703c89aa5d51c7e0b987caa7c6ad9aa837375577289ce\": container with ID starting with 7528863bf70b705a0ea703c89aa5d51c7e0b987caa7c6ad9aa837375577289ce not found: ID does not exist" Jul 30 01:30:01.395818 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.395705 2564 scope.go:117] "RemoveContainer" containerID="08c68ca4476c0c82cda00c6fead6d8954d6f1042cb9b96285126f7f58c8b9af8" Jul 30 01:30:01.396032 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:30:01.396011 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08c68ca4476c0c82cda00c6fead6d8954d6f1042cb9b96285126f7f58c8b9af8\": container with ID starting with 08c68ca4476c0c82cda00c6fead6d8954d6f1042cb9b96285126f7f58c8b9af8 not found: ID does not exist" containerID="08c68ca4476c0c82cda00c6fead6d8954d6f1042cb9b96285126f7f58c8b9af8" Jul 30 01:30:01.396104 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.396046 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08c68ca4476c0c82cda00c6fead6d8954d6f1042cb9b96285126f7f58c8b9af8"} err="failed to get container status \"08c68ca4476c0c82cda00c6fead6d8954d6f1042cb9b96285126f7f58c8b9af8\": rpc error: code = NotFound desc = could not find container \"08c68ca4476c0c82cda00c6fead6d8954d6f1042cb9b96285126f7f58c8b9af8\": container with ID starting with 08c68ca4476c0c82cda00c6fead6d8954d6f1042cb9b96285126f7f58c8b9af8 not found: ID does not exist" Jul 30 01:30:01.488197 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.488158 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:30:01.494342 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.494308 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-podinfo\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:30:01.494342 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.494339 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istiod-ca-cert\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:30:01.494342 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.494348 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-credential-socket\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:30:01.494569 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.494363 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-data\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:30:01.494569 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.494375 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7mpg\" (UniqueName: \"kubernetes.io/projected/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-kube-api-access-n7mpg\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:30:01.494569 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.494383 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-envoy\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:30:01.494569 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.494394 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc-istio-token\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:30:01.587620 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.587583 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:30:01.697153 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.697079 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt"] Jul 30 01:30:01.701428 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:01.701388 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-b2ddd3-openshift-default-5645956f74-c45dt"] Jul 30 01:30:03.669376 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:03.669344 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" path="/var/lib/kubelet/pods/f644aece-e0e8-4abd-83ed-ca7f5ddca5fc/volumes" Jul 30 01:30:24.839287 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:24.839247 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:30:25.324043 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.324005 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:30:25.589860 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.589768 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx"] Jul 30 01:30:25.590634 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.590607 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e6c7714-60fb-4cfb-b210-8131be6fc03a" containerName="revoke-keys" Jul 30 01:30:25.590772 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.590761 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e6c7714-60fb-4cfb-b210-8131be6fc03a" containerName="revoke-keys" Jul 30 01:30:25.590859 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.590850 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" containerName="istio-proxy" Jul 30 01:30:25.590953 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.590944 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" containerName="istio-proxy" Jul 30 01:30:25.591059 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.591050 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" containerName="istio-proxy" Jul 30 01:30:25.591133 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.591124 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" containerName="istio-proxy" Jul 30 01:30:25.591309 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.591299 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" containerName="istio-proxy" Jul 30 01:30:25.591389 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.591380 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e6c7714-60fb-4cfb-b210-8131be6fc03a" containerName="revoke-keys" Jul 30 01:30:25.591468 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.591459 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f644aece-e0e8-4abd-83ed-ca7f5ddca5fc" containerName="istio-proxy" Jul 30 01:30:25.603067 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.603036 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.605923 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.605875 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-b11aa3-openshift-default-dockercfg-9q9bl\"" Jul 30 01:30:25.606126 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.606103 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx"] Jul 30 01:30:25.615040 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.615014 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:30:25.731911 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.731871 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-workload-certs\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.732131 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.731946 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-token\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.732131 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.732047 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-podinfo\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.732131 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.732093 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-envoy\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.732310 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.732145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d44e878f-0b55-453e-b5e2-6ab4ac402202-istiod-ca-cert\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.732310 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.732201 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-workload-socket\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.732310 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.732257 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvrd2\" (UniqueName: \"kubernetes.io/projected/d44e878f-0b55-453e-b5e2-6ab4ac402202-kube-api-access-nvrd2\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.732310 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.732301 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-credential-socket\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.732489 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.732336 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-data\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.788430 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.788394 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:30:25.833190 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.833145 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-workload-certs\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.833190 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.833187 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-token\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.833462 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.833231 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-podinfo\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.833462 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.833256 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-envoy\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.833462 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.833278 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d44e878f-0b55-453e-b5e2-6ab4ac402202-istiod-ca-cert\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.833462 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.833328 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-workload-socket\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.833462 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.833377 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvrd2\" (UniqueName: \"kubernetes.io/projected/d44e878f-0b55-453e-b5e2-6ab4ac402202-kube-api-access-nvrd2\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.833462 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.833419 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-credential-socket\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.833462 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.833442 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-data\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.833805 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.833620 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-workload-certs\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.833805 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.833787 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-workload-socket\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.833805 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.833798 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-data\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.834011 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.833992 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-credential-socket\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.834173 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.834113 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d44e878f-0b55-453e-b5e2-6ab4ac402202-istiod-ca-cert\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.835694 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.835670 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-envoy\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.835816 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.835801 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-podinfo\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.840937 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.840855 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvrd2\" (UniqueName: \"kubernetes.io/projected/d44e878f-0b55-453e-b5e2-6ab4ac402202-kube-api-access-nvrd2\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.840937 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.840919 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-token\") pod \"e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:25.915040 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:25.914995 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:26.060884 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:26.060854 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx"] Jul 30 01:30:26.063506 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:30:26.063477 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd44e878f_0b55_453e_b5e2_6ab4ac402202.slice/crio-25f38efbac4fd89080b5969e1cd32861cf86f133c6e257843c4612b7560ddbd8 WatchSource:0}: Error finding container 25f38efbac4fd89080b5969e1cd32861cf86f133c6e257843c4612b7560ddbd8: Status 404 returned error can't find the container with id 25f38efbac4fd89080b5969e1cd32861cf86f133c6e257843c4612b7560ddbd8 Jul 30 01:30:26.065688 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:26.065654 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:30:26.065798 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:26.065737 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:30:26.065798 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:26.065785 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:30:26.522241 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:26.522150 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" event={"ID":"d44e878f-0b55-453e-b5e2-6ab4ac402202","Type":"ContainerStarted","Data":"f2a0a999ba87f3a4de4bf22a2bab9f63d25a3c99e94201efa2993b05a4467866"} Jul 30 01:30:26.522241 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:26.522204 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" event={"ID":"d44e878f-0b55-453e-b5e2-6ab4ac402202","Type":"ContainerStarted","Data":"25f38efbac4fd89080b5969e1cd32861cf86f133c6e257843c4612b7560ddbd8"} Jul 30 01:30:26.546088 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:26.546023 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" podStartSLOduration=1.546001401 podStartE2EDuration="1.546001401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:30:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:30:26.541371081 +0000 UTC m=+2829.342328185" watchObservedRunningTime="2026-07-30 01:30:26.546001401 +0000 UTC m=+2829.346958508" Jul 30 01:30:26.915621 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:26.915568 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:27.920096 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:27.920068 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:28.464196 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:28.464161 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:30:28.534636 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:28.534605 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:28.535576 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:28.535555 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:28.560891 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:28.560846 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:30:29.654619 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:29.654558 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:30:29.856419 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:29.856385 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:30:30.238789 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:30.238723 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:30:30.347373 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:30.347343 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:30:44.576095 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:44.576051 2564 patch_prober.go:28] interesting pod/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 30 01:30:44.576529 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:44.576120 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" podUID="d44e878f-0b55-453e-b5e2-6ab4ac402202" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 30 01:30:45.637435 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:45.637405 2564 generic.go:356] "Generic (PLEG): container finished" podID="d44e878f-0b55-453e-b5e2-6ab4ac402202" containerID="f2a0a999ba87f3a4de4bf22a2bab9f63d25a3c99e94201efa2993b05a4467866" exitCode=0 Jul 30 01:30:45.637875 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:45.637485 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" event={"ID":"d44e878f-0b55-453e-b5e2-6ab4ac402202","Type":"ContainerDied","Data":"f2a0a999ba87f3a4de4bf22a2bab9f63d25a3c99e94201efa2993b05a4467866"} Jul 30 01:30:45.637957 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:45.637942 2564 scope.go:117] "RemoveContainer" containerID="f2a0a999ba87f3a4de4bf22a2bab9f63d25a3c99e94201efa2993b05a4467866" Jul 30 01:30:45.638980 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:45.638951 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:30:45.639087 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:45.639043 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:30:45.639156 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:45.639098 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:30:46.645719 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:46.645683 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" event={"ID":"d44e878f-0b55-453e-b5e2-6ab4ac402202","Type":"ContainerStarted","Data":"dcd9aacab746483f105dda01cefa59dab3e3c7416aa50736a33768036a2b811a"} Jul 30 01:30:46.916514 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:46.916401 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:46.916702 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:46.916579 2564 patch_prober.go:28] interesting pod/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 30 01:30:46.916702 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:46.916628 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" podUID="d44e878f-0b55-453e-b5e2-6ab4ac402202" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 30 01:30:47.916331 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:47.916285 2564 patch_prober.go:28] interesting pod/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 30 01:30:47.916988 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:47.916949 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" podUID="d44e878f-0b55-453e-b5e2-6ab4ac402202" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 30 01:30:48.916050 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:48.916004 2564 patch_prober.go:28] interesting pod/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 30 01:30:48.916279 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:48.916092 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" podUID="d44e878f-0b55-453e-b5e2-6ab4ac402202" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 30 01:30:49.915785 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:49.915745 2564 patch_prober.go:28] interesting pod/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 30 01:30:49.916414 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:49.916379 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" podUID="d44e878f-0b55-453e-b5e2-6ab4ac402202" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 30 01:30:50.919807 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:50.919775 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:50.920267 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:50.920115 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:50.920867 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:50.920850 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:30:53.439160 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:30:53.439122 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:10.090939 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:10.090818 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:12.001012 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:12.000964 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:12.104282 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:12.104242 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:12.168334 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:12.168292 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:12.558402 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:12.558371 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:18.368914 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:18.368867 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:18.469751 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:18.469706 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:18.636054 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:18.635957 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:18.868477 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:18.868439 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:34.432222 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:34.432180 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:34.500288 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:34.500255 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:34.571149 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:34.571113 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:34.627410 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:34.627375 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:34.995483 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:34.995448 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:40.783780 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:40.783745 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:40.885446 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:40.885409 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:41.031779 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:41.031745 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx"] Jul 30 01:31:41.032010 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:41.031985 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" podUID="d44e878f-0b55-453e-b5e2-6ab4ac402202" containerName="istio-proxy" containerID="cri-o://dcd9aacab746483f105dda01cefa59dab3e3c7416aa50736a33768036a2b811a" gracePeriod=30 Jul 30 01:31:41.054088 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:41.053993 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:41.279093 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:41.279051 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:41.650328 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:41.650291 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:41.939853 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:41.939769 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:42.122398 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:42.122361 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:45.362024 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:45.361985 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:45.465787 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:45.465750 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:46.288968 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.288943 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:31:46.423295 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.423256 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-data\") pod \"d44e878f-0b55-453e-b5e2-6ab4ac402202\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " Jul 30 01:31:46.423295 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.423294 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-credential-socket\") pod \"d44e878f-0b55-453e-b5e2-6ab4ac402202\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " Jul 30 01:31:46.423814 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.423318 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-workload-certs\") pod \"d44e878f-0b55-453e-b5e2-6ab4ac402202\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " Jul 30 01:31:46.423814 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.423359 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d44e878f-0b55-453e-b5e2-6ab4ac402202-istiod-ca-cert\") pod \"d44e878f-0b55-453e-b5e2-6ab4ac402202\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " Jul 30 01:31:46.423814 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.423399 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-podinfo\") pod \"d44e878f-0b55-453e-b5e2-6ab4ac402202\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " Jul 30 01:31:46.423814 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.423444 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-workload-socket\") pod \"d44e878f-0b55-453e-b5e2-6ab4ac402202\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " Jul 30 01:31:46.423814 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.423467 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-envoy\") pod \"d44e878f-0b55-453e-b5e2-6ab4ac402202\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " Jul 30 01:31:46.423814 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.423519 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvrd2\" (UniqueName: \"kubernetes.io/projected/d44e878f-0b55-453e-b5e2-6ab4ac402202-kube-api-access-nvrd2\") pod \"d44e878f-0b55-453e-b5e2-6ab4ac402202\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " Jul 30 01:31:46.423814 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.423569 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-token\") pod \"d44e878f-0b55-453e-b5e2-6ab4ac402202\" (UID: \"d44e878f-0b55-453e-b5e2-6ab4ac402202\") " Jul 30 01:31:46.423814 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.423580 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d44e878f-0b55-453e-b5e2-6ab4ac402202" (UID: "d44e878f-0b55-453e-b5e2-6ab4ac402202"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:31:46.423814 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.423664 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d44e878f-0b55-453e-b5e2-6ab4ac402202" (UID: "d44e878f-0b55-453e-b5e2-6ab4ac402202"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:31:46.423814 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.423766 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d44e878f-0b55-453e-b5e2-6ab4ac402202" (UID: "d44e878f-0b55-453e-b5e2-6ab4ac402202"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:31:46.424310 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.423969 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-workload-certs\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:31:46.424310 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.423981 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d44e878f-0b55-453e-b5e2-6ab4ac402202-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d44e878f-0b55-453e-b5e2-6ab4ac402202" (UID: "d44e878f-0b55-453e-b5e2-6ab4ac402202"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:31:46.424310 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.423992 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-workload-socket\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:31:46.424310 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.424028 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-credential-socket\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:31:46.424310 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.424190 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-data" (OuterVolumeSpecName: "istio-data") pod "d44e878f-0b55-453e-b5e2-6ab4ac402202" (UID: "d44e878f-0b55-453e-b5e2-6ab4ac402202"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:31:46.425840 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.425811 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d44e878f-0b55-453e-b5e2-6ab4ac402202" (UID: "d44e878f-0b55-453e-b5e2-6ab4ac402202"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:31:46.425981 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.425952 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d44e878f-0b55-453e-b5e2-6ab4ac402202-kube-api-access-nvrd2" (OuterVolumeSpecName: "kube-api-access-nvrd2") pod "d44e878f-0b55-453e-b5e2-6ab4ac402202" (UID: "d44e878f-0b55-453e-b5e2-6ab4ac402202"). InnerVolumeSpecName "kube-api-access-nvrd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:31:46.426216 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.426193 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d44e878f-0b55-453e-b5e2-6ab4ac402202" (UID: "d44e878f-0b55-453e-b5e2-6ab4ac402202"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 01:31:46.426296 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.426221 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-token" (OuterVolumeSpecName: "istio-token") pod "d44e878f-0b55-453e-b5e2-6ab4ac402202" (UID: "d44e878f-0b55-453e-b5e2-6ab4ac402202"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:31:46.525380 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.525339 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-envoy\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:31:46.525380 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.525372 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvrd2\" (UniqueName: \"kubernetes.io/projected/d44e878f-0b55-453e-b5e2-6ab4ac402202-kube-api-access-nvrd2\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:31:46.525380 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.525385 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-token\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:31:46.525668 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.525395 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-data\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:31:46.525668 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.525406 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d44e878f-0b55-453e-b5e2-6ab4ac402202-istiod-ca-cert\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:31:46.525668 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.525417 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d44e878f-0b55-453e-b5e2-6ab4ac402202-istio-podinfo\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:31:46.597363 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.597313 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:46.695663 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.695570 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:46.976863 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.976769 2564 generic.go:356] "Generic (PLEG): container finished" podID="d44e878f-0b55-453e-b5e2-6ab4ac402202" containerID="dcd9aacab746483f105dda01cefa59dab3e3c7416aa50736a33768036a2b811a" exitCode=0 Jul 30 01:31:46.976863 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.976834 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" Jul 30 01:31:46.976863 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.976850 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" event={"ID":"d44e878f-0b55-453e-b5e2-6ab4ac402202","Type":"ContainerDied","Data":"dcd9aacab746483f105dda01cefa59dab3e3c7416aa50736a33768036a2b811a"} Jul 30 01:31:46.977095 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.976921 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx" event={"ID":"d44e878f-0b55-453e-b5e2-6ab4ac402202","Type":"ContainerDied","Data":"25f38efbac4fd89080b5969e1cd32861cf86f133c6e257843c4612b7560ddbd8"} Jul 30 01:31:46.977095 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.976945 2564 scope.go:117] "RemoveContainer" containerID="dcd9aacab746483f105dda01cefa59dab3e3c7416aa50736a33768036a2b811a" Jul 30 01:31:46.987452 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.987430 2564 scope.go:117] "RemoveContainer" containerID="f2a0a999ba87f3a4de4bf22a2bab9f63d25a3c99e94201efa2993b05a4467866" Jul 30 01:31:46.996476 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.996374 2564 scope.go:117] "RemoveContainer" containerID="dcd9aacab746483f105dda01cefa59dab3e3c7416aa50736a33768036a2b811a" Jul 30 01:31:46.996671 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:31:46.996652 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcd9aacab746483f105dda01cefa59dab3e3c7416aa50736a33768036a2b811a\": container with ID starting with dcd9aacab746483f105dda01cefa59dab3e3c7416aa50736a33768036a2b811a not found: ID does not exist" containerID="dcd9aacab746483f105dda01cefa59dab3e3c7416aa50736a33768036a2b811a" Jul 30 01:31:46.996746 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.996683 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcd9aacab746483f105dda01cefa59dab3e3c7416aa50736a33768036a2b811a"} err="failed to get container status \"dcd9aacab746483f105dda01cefa59dab3e3c7416aa50736a33768036a2b811a\": rpc error: code = NotFound desc = could not find container \"dcd9aacab746483f105dda01cefa59dab3e3c7416aa50736a33768036a2b811a\": container with ID starting with dcd9aacab746483f105dda01cefa59dab3e3c7416aa50736a33768036a2b811a not found: ID does not exist" Jul 30 01:31:46.996746 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.996708 2564 scope.go:117] "RemoveContainer" containerID="f2a0a999ba87f3a4de4bf22a2bab9f63d25a3c99e94201efa2993b05a4467866" Jul 30 01:31:46.997022 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:31:46.997003 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2a0a999ba87f3a4de4bf22a2bab9f63d25a3c99e94201efa2993b05a4467866\": container with ID starting with f2a0a999ba87f3a4de4bf22a2bab9f63d25a3c99e94201efa2993b05a4467866 not found: ID does not exist" containerID="f2a0a999ba87f3a4de4bf22a2bab9f63d25a3c99e94201efa2993b05a4467866" Jul 30 01:31:46.997069 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:46.997029 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a0a999ba87f3a4de4bf22a2bab9f63d25a3c99e94201efa2993b05a4467866"} err="failed to get container status \"f2a0a999ba87f3a4de4bf22a2bab9f63d25a3c99e94201efa2993b05a4467866\": rpc error: code = NotFound desc = could not find container \"f2a0a999ba87f3a4de4bf22a2bab9f63d25a3c99e94201efa2993b05a4467866\": container with ID starting with f2a0a999ba87f3a4de4bf22a2bab9f63d25a3c99e94201efa2993b05a4467866 not found: ID does not exist" Jul 30 01:31:47.003943 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:47.003916 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx"] Jul 30 01:31:47.007696 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:47.007673 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-b11aa3-openshift-default-79d474bbd8-5glqx"] Jul 30 01:31:47.011470 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:31:47.011439 2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd44e878f_0b55_453e_b5e2_6ab4ac402202.slice/crio-25f38efbac4fd89080b5969e1cd32861cf86f133c6e257843c4612b7560ddbd8\": RecentStats: unable to find data in memory cache]" Jul 30 01:31:47.088421 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:47.088383 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:47.182596 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:47.182558 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:31:47.671689 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:31:47.671483 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d44e878f-0b55-453e-b5e2-6ab4ac402202" path="/var/lib/kubelet/pods/d44e878f-0b55-453e-b5e2-6ab4ac402202/volumes" Jul 30 01:32:10.360306 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:10.360265 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:32:10.685099 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:10.684991 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:32:10.970762 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:10.970687 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg"] Jul 30 01:32:10.971104 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:10.971090 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d44e878f-0b55-453e-b5e2-6ab4ac402202" containerName="istio-proxy" Jul 30 01:32:10.971165 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:10.971105 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d44e878f-0b55-453e-b5e2-6ab4ac402202" containerName="istio-proxy" Jul 30 01:32:10.971165 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:10.971116 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d44e878f-0b55-453e-b5e2-6ab4ac402202" containerName="istio-proxy" Jul 30 01:32:10.971165 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:10.971121 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d44e878f-0b55-453e-b5e2-6ab4ac402202" containerName="istio-proxy" Jul 30 01:32:10.971258 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:10.971193 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d44e878f-0b55-453e-b5e2-6ab4ac402202" containerName="istio-proxy" Jul 30 01:32:10.971258 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:10.971206 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d44e878f-0b55-453e-b5e2-6ab4ac402202" containerName="istio-proxy" Jul 30 01:32:10.974421 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:10.974405 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:10.977046 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:10.977030 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-fba93c-openshift-default-dockercfg-8slwd\"" Jul 30 01:32:10.991301 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:10.991264 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg"] Jul 30 01:32:10.998577 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:10.998540 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:32:11.048240 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.048207 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-istio-data\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.048240 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.048243 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c18af419-ea76-4774-b5b2-608b96926159-istio-podinfo\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.048444 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.048269 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-credential-socket\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.048444 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.048381 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-workload-socket\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.048444 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.048413 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c18af419-ea76-4774-b5b2-608b96926159-istiod-ca-cert\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.048444 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.048434 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzrwf\" (UniqueName: \"kubernetes.io/projected/c18af419-ea76-4774-b5b2-608b96926159-kube-api-access-lzrwf\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.048575 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.048458 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c18af419-ea76-4774-b5b2-608b96926159-istio-token\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.048575 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.048521 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-workload-certs\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.048575 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.048543 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-istio-envoy\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.149333 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.149297 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-workload-socket\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.149333 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.149340 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c18af419-ea76-4774-b5b2-608b96926159-istiod-ca-cert\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.149585 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.149369 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzrwf\" (UniqueName: \"kubernetes.io/projected/c18af419-ea76-4774-b5b2-608b96926159-kube-api-access-lzrwf\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.149585 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.149400 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c18af419-ea76-4774-b5b2-608b96926159-istio-token\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.149585 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.149456 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-workload-certs\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.149585 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.149492 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-istio-envoy\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.149585 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.149527 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-istio-data\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.149585 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.149553 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c18af419-ea76-4774-b5b2-608b96926159-istio-podinfo\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.149982 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.149588 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-credential-socket\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.149982 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.149746 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-workload-socket\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.149982 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.149870 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-workload-certs\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.149982 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.149929 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-istio-data\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.150208 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.150045 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-credential-socket\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.150260 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.150207 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c18af419-ea76-4774-b5b2-608b96926159-istiod-ca-cert\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.151877 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.151857 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-istio-envoy\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.152041 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.152024 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c18af419-ea76-4774-b5b2-608b96926159-istio-podinfo\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.156820 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.156791 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c18af419-ea76-4774-b5b2-608b96926159-istio-token\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.157010 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.156991 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzrwf\" (UniqueName: \"kubernetes.io/projected/c18af419-ea76-4774-b5b2-608b96926159-kube-api-access-lzrwf\") pod \"e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.211226 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.211186 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:32:11.287157 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.287069 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:11.428828 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.428796 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg"] Jul 30 01:32:11.430636 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:32:11.430605 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc18af419_ea76_4774_b5b2_608b96926159.slice/crio-f2b368b75fd9711442442a1a2f20e4fa699b4b408b2c5dede6ba786162ae889b WatchSource:0}: Error finding container f2b368b75fd9711442442a1a2f20e4fa699b4b408b2c5dede6ba786162ae889b: Status 404 returned error can't find the container with id f2b368b75fd9711442442a1a2f20e4fa699b4b408b2c5dede6ba786162ae889b Jul 30 01:32:11.433089 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.433057 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:32:11.433195 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.433122 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:32:11.433195 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:11.433162 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:32:12.117189 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:12.117140 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" event={"ID":"c18af419-ea76-4774-b5b2-608b96926159","Type":"ContainerStarted","Data":"0b2e5e9a38c7d9567428417335020cff6355ad8e1d7ac7ab50ebc93446e0aad7"} Jul 30 01:32:12.117369 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:12.117200 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" event={"ID":"c18af419-ea76-4774-b5b2-608b96926159","Type":"ContainerStarted","Data":"f2b368b75fd9711442442a1a2f20e4fa699b4b408b2c5dede6ba786162ae889b"} Jul 30 01:32:12.137878 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:12.137822 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" podStartSLOduration=2.137803206 podStartE2EDuration="2.137803206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:32:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:32:12.13526253 +0000 UTC m=+2934.936219638" watchObservedRunningTime="2026-07-30 01:32:12.137803206 +0000 UTC m=+2934.938760312" Jul 30 01:32:12.287675 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:12.287634 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:12.293276 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:12.293249 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:13.123265 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:13.123233 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:13.124181 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:13.124164 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:14.061699 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:14.061657 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:32:14.229480 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:14.229450 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:32:15.236563 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:15.236529 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:32:15.369238 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:15.369196 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:32:15.708147 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:15.708117 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:32:15.810008 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:15.809978 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:32:29.165039 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:29.164999 2564 patch_prober.go:28] interesting pod/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 30 01:32:29.165460 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:29.165059 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" podUID="c18af419-ea76-4774-b5b2-608b96926159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 30 01:32:30.219272 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:30.219177 2564 generic.go:356] "Generic (PLEG): container finished" podID="c18af419-ea76-4774-b5b2-608b96926159" containerID="0b2e5e9a38c7d9567428417335020cff6355ad8e1d7ac7ab50ebc93446e0aad7" exitCode=0 Jul 30 01:32:30.219272 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:30.219250 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" event={"ID":"c18af419-ea76-4774-b5b2-608b96926159","Type":"ContainerDied","Data":"0b2e5e9a38c7d9567428417335020cff6355ad8e1d7ac7ab50ebc93446e0aad7"} Jul 30 01:32:30.219714 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:30.219697 2564 scope.go:117] "RemoveContainer" containerID="0b2e5e9a38c7d9567428417335020cff6355ad8e1d7ac7ab50ebc93446e0aad7" Jul 30 01:32:30.220774 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:30.220746 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:32:30.220882 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:30.220849 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:32:30.220968 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:30.220918 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:32:31.228693 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:31.228661 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" event={"ID":"c18af419-ea76-4774-b5b2-608b96926159","Type":"ContainerStarted","Data":"d8c18cc9a64b2274ae4da3c15898ed017b8a1e7510e55027058a1cf91da8a864"} Jul 30 01:32:31.288073 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:31.288039 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:31.288276 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:31.288219 2564 patch_prober.go:28] interesting pod/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 30 01:32:31.288335 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:31.288267 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" podUID="c18af419-ea76-4774-b5b2-608b96926159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 30 01:32:32.288033 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:32.288000 2564 patch_prober.go:28] interesting pod/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 30 01:32:32.288436 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:32.288050 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" podUID="c18af419-ea76-4774-b5b2-608b96926159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 30 01:32:33.287714 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:33.287671 2564 patch_prober.go:28] interesting pod/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 30 01:32:33.287929 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:33.287736 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" podUID="c18af419-ea76-4774-b5b2-608b96926159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 30 01:32:34.288495 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:34.288448 2564 patch_prober.go:28] interesting pod/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 30 01:32:34.288887 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:34.288509 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" podUID="c18af419-ea76-4774-b5b2-608b96926159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 30 01:32:35.291439 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:35.291406 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:35.291945 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:35.291845 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:35.292541 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:35.292524 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:32:39.082998 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:39.082888 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:32:45.974542 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:32:45.974502 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:33:19.677028 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:33:19.676990 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:33:40.978163 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:33:40.978128 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:35:16.714855 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:35:16.714816 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:36:04.628065 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:04.627962 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:36:10.964420 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:10.964385 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:36:14.108964 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:14.108928 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cpr82"] Jul 30 01:36:14.112618 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:14.112599 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cpr82" Jul 30 01:36:14.115202 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:14.115173 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-b6ljj\"" Jul 30 01:36:14.115339 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:14.115245 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 30 01:36:14.116068 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:14.116051 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 30 01:36:14.131542 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:14.131509 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cpr82"] Jul 30 01:36:14.228218 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:14.228177 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smn28\" (UniqueName: \"kubernetes.io/projected/b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c-kube-api-access-smn28\") pod \"kuadrant-operator-catalog-cpr82\" (UID: \"b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c\") " pod="kuadrant-system/kuadrant-operator-catalog-cpr82" Jul 30 01:36:14.329719 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:14.329679 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smn28\" (UniqueName: \"kubernetes.io/projected/b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c-kube-api-access-smn28\") pod \"kuadrant-operator-catalog-cpr82\" (UID: \"b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c\") " pod="kuadrant-system/kuadrant-operator-catalog-cpr82" Jul 30 01:36:14.338916 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:14.338863 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smn28\" (UniqueName: \"kubernetes.io/projected/b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c-kube-api-access-smn28\") pod \"kuadrant-operator-catalog-cpr82\" (UID: \"b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c\") " pod="kuadrant-system/kuadrant-operator-catalog-cpr82" Jul 30 01:36:14.422482 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:14.422389 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cpr82" Jul 30 01:36:14.564625 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:14.564596 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cpr82"] Jul 30 01:36:14.566945 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:36:14.566913 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2688d8c_8f15_4a8c_b09e_fc596fb1ac4c.slice/crio-246bef617384e1b5e4c87b90a8a3dc7326272a824914fceeb0d48295ab51e2ce WatchSource:0}: Error finding container 246bef617384e1b5e4c87b90a8a3dc7326272a824914fceeb0d48295ab51e2ce: Status 404 returned error can't find the container with id 246bef617384e1b5e4c87b90a8a3dc7326272a824914fceeb0d48295ab51e2ce Jul 30 01:36:14.568442 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:14.568421 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 01:36:15.468753 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:15.468713 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cpr82" event={"ID":"b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c","Type":"ContainerStarted","Data":"246bef617384e1b5e4c87b90a8a3dc7326272a824914fceeb0d48295ab51e2ce"} Jul 30 01:36:17.481971 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:17.481930 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cpr82" event={"ID":"b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c","Type":"ContainerStarted","Data":"b0476b7946bf225eb9c16f4de634a0327b1fd57075c71a725ecdc2c1f09d13cd"} Jul 30 01:36:17.499452 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:17.499401 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-cpr82" podStartSLOduration=1.508627604 podStartE2EDuration="3.499384124s" totalImagesPullingTime="1.99075652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:36:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:36:17.496839772 +0000 UTC m=+3180.297796889" watchObservedRunningTime="2026-07-30 01:36:17.499384124 +0000 UTC m=+3180.300341230" Jul 30 01:36:17.991989 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:17.991950 2564 scope.go:117] "RemoveContainer" containerID="dd353b795f470c4862c09e948218db44e4ae340cc8dc16620b10460cb39278af" Jul 30 01:36:24.423498 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:24.423392 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-cpr82" Jul 30 01:36:24.423888 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:24.423509 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-cpr82" Jul 30 01:36:24.445644 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:24.445611 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-cpr82" Jul 30 01:36:24.543351 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:24.543320 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-cpr82" Jul 30 01:36:24.672247 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:24.672200 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cpr82"] Jul 30 01:36:26.531860 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:26.531797 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-cpr82" podUID="b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c" containerName="registry-server" containerID="cri-o://b0476b7946bf225eb9c16f4de634a0327b1fd57075c71a725ecdc2c1f09d13cd" gracePeriod=2 Jul 30 01:36:26.792257 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:26.792185 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cpr82" Jul 30 01:36:26.959928 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:26.959885 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smn28\" (UniqueName: \"kubernetes.io/projected/b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c-kube-api-access-smn28\") pod \"b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c\" (UID: \"b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c\") " Jul 30 01:36:26.961958 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:26.961930 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c-kube-api-access-smn28" (OuterVolumeSpecName: "kube-api-access-smn28") pod "b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c" (UID: "b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c"). InnerVolumeSpecName "kube-api-access-smn28". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:36:27.060700 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:27.060598 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smn28\" (UniqueName: \"kubernetes.io/projected/b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c-kube-api-access-smn28\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:36:27.538263 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:27.538228 2564 generic.go:356] "Generic (PLEG): container finished" podID="b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c" containerID="b0476b7946bf225eb9c16f4de634a0327b1fd57075c71a725ecdc2c1f09d13cd" exitCode=0 Jul 30 01:36:27.538694 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:27.538276 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cpr82" Jul 30 01:36:27.538694 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:27.538316 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cpr82" event={"ID":"b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c","Type":"ContainerDied","Data":"b0476b7946bf225eb9c16f4de634a0327b1fd57075c71a725ecdc2c1f09d13cd"} Jul 30 01:36:27.538694 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:27.538367 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cpr82" event={"ID":"b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c","Type":"ContainerDied","Data":"246bef617384e1b5e4c87b90a8a3dc7326272a824914fceeb0d48295ab51e2ce"} Jul 30 01:36:27.538694 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:27.538387 2564 scope.go:117] "RemoveContainer" containerID="b0476b7946bf225eb9c16f4de634a0327b1fd57075c71a725ecdc2c1f09d13cd" Jul 30 01:36:27.550330 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:27.550310 2564 scope.go:117] "RemoveContainer" containerID="b0476b7946bf225eb9c16f4de634a0327b1fd57075c71a725ecdc2c1f09d13cd" Jul 30 01:36:27.550663 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:36:27.550644 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0476b7946bf225eb9c16f4de634a0327b1fd57075c71a725ecdc2c1f09d13cd\": container with ID starting with b0476b7946bf225eb9c16f4de634a0327b1fd57075c71a725ecdc2c1f09d13cd not found: ID does not exist" containerID="b0476b7946bf225eb9c16f4de634a0327b1fd57075c71a725ecdc2c1f09d13cd" Jul 30 01:36:27.550713 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:27.550674 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0476b7946bf225eb9c16f4de634a0327b1fd57075c71a725ecdc2c1f09d13cd"} err="failed to get container status \"b0476b7946bf225eb9c16f4de634a0327b1fd57075c71a725ecdc2c1f09d13cd\": rpc error: code = NotFound desc = could not find container \"b0476b7946bf225eb9c16f4de634a0327b1fd57075c71a725ecdc2c1f09d13cd\": container with ID starting with b0476b7946bf225eb9c16f4de634a0327b1fd57075c71a725ecdc2c1f09d13cd not found: ID does not exist" Jul 30 01:36:27.564235 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:27.564201 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cpr82"] Jul 30 01:36:27.567648 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:27.567619 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cpr82"] Jul 30 01:36:27.670233 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:36:27.670201 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c" path="/var/lib/kubelet/pods/b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c/volumes" Jul 30 01:38:09.691978 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:38:09.691927 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:38:10.602718 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:38:10.602684 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:38:24.673014 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:38:24.672976 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:38:31.322700 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:38:31.322663 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:38:31.379761 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:38:31.379728 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:38:59.690157 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:38:59.690063 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:40:36.301077 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:40:36.301041 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:40:37.198331 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:40:37.198297 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:40:51.365252 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:40:51.365217 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:40:56.562016 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:40:56.561975 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:41:07.079173 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:41:07.079134 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:42:48.786332 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:42:48.786295 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:42:49.948829 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:42:49.948797 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:09.369136 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:09.369092 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:09.659923 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:09.659803 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:09.839820 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:09.839788 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:12.640566 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:12.640524 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:12.741647 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:12.741613 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:13.764127 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:13.764089 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:14.159596 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:14.159566 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:14.273378 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:14.273337 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:14.364676 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:14.364641 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:37.618761 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:37.618720 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:38.274019 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:38.273980 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:38.576033 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:38.575999 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:38.849909 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:38.849804 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:40.900388 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:40.900358 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:40.954382 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:40.954350 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:41.759579 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:41.759544 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:41.862083 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:41.862052 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:42.855768 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:42.855731 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz"] Jul 30 01:43:42.856172 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:42.856158 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c" containerName="registry-server" Jul 30 01:43:42.856172 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:42.856170 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c" containerName="registry-server" Jul 30 01:43:42.856253 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:42.856234 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2688d8c-8f15-4a8c-b09e-fc596fb1ac4c" containerName="registry-server" Jul 30 01:43:42.859514 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:42.859492 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:43:42.861673 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:42.861649 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-b64c39-serving-cert\"" Jul 30 01:43:42.861845 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:42.861782 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 30 01:43:42.861845 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:42.861822 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dnc2x\"" Jul 30 01:43:42.862699 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:42.862677 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 30 01:43:42.868781 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:42.868517 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz"] Jul 30 01:43:42.891068 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:42.891035 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:42.909061 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:42.909028 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-457hl\" (UniqueName: \"kubernetes.io/projected/92472797-0d13-4e6f-b636-9949b5e9f238-kube-api-access-457hl\") pod \"maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz\" (UID: \"92472797-0d13-4e6f-b636-9949b5e9f238\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:43:42.909263 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:42.909090 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92472797-0d13-4e6f-b636-9949b5e9f238-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz\" (UID: \"92472797-0d13-4e6f-b636-9949b5e9f238\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:43:42.909263 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:42.909153 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92472797-0d13-4e6f-b636-9949b5e9f238-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz\" (UID: \"92472797-0d13-4e6f-b636-9949b5e9f238\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:43:43.010336 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.010288 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-457hl\" (UniqueName: \"kubernetes.io/projected/92472797-0d13-4e6f-b636-9949b5e9f238-kube-api-access-457hl\") pod \"maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz\" (UID: \"92472797-0d13-4e6f-b636-9949b5e9f238\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:43:43.010538 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.010364 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92472797-0d13-4e6f-b636-9949b5e9f238-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz\" (UID: \"92472797-0d13-4e6f-b636-9949b5e9f238\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:43:43.010538 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.010408 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92472797-0d13-4e6f-b636-9949b5e9f238-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz\" (UID: \"92472797-0d13-4e6f-b636-9949b5e9f238\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:43:43.010661 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:43:43.010587 2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-serving-cert: secret "maas-api-e2e-ipp-b-b64c39-serving-cert" not found Jul 30 01:43:43.010721 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:43:43.010686 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92472797-0d13-4e6f-b636-9949b5e9f238-maas-api-tls podName:92472797-0d13-4e6f-b636-9949b5e9f238 nodeName:}" failed. No retries permitted until 2026-07-30 01:43:43.510653199 +0000 UTC m=+3626.311610300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/92472797-0d13-4e6f-b636-9949b5e9f238-maas-api-tls") pod "maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" (UID: "92472797-0d13-4e6f-b636-9949b5e9f238") : secret "maas-api-e2e-ipp-b-b64c39-serving-cert" not found Jul 30 01:43:43.011039 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.011017 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92472797-0d13-4e6f-b636-9949b5e9f238-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz\" (UID: \"92472797-0d13-4e6f-b636-9949b5e9f238\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:43:43.020288 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.020256 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-457hl\" (UniqueName: \"kubernetes.io/projected/92472797-0d13-4e6f-b636-9949b5e9f238-kube-api-access-457hl\") pod \"maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz\" (UID: \"92472797-0d13-4e6f-b636-9949b5e9f238\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:43:43.329986 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.329950 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:43.426961 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.426925 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:43.487116 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.487074 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td"] Jul 30 01:43:43.490718 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.490700 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" Jul 30 01:43:43.493106 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.493083 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-b64c39\"" Jul 30 01:43:43.493106 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.493096 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-b64c39-dockercfg-p99xt\"" Jul 30 01:43:43.501812 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.501772 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td"] Jul 30 01:43:43.508109 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.508077 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:43:43.516168 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.516135 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92472797-0d13-4e6f-b636-9949b5e9f238-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz\" (UID: \"92472797-0d13-4e6f-b636-9949b5e9f238\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:43:43.516372 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.516272 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbqbq\" (UniqueName: \"kubernetes.io/projected/558c0c9a-d249-4941-9ec6-ef7df138048c-kube-api-access-wbqbq\") pod \"payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td\" (UID: \"558c0c9a-d249-4941-9ec6-ef7df138048c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" Jul 30 01:43:43.516372 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.516306 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/558c0c9a-d249-4941-9ec6-ef7df138048c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td\" (UID: \"558c0c9a-d249-4941-9ec6-ef7df138048c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" Jul 30 01:43:43.519161 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.519136 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92472797-0d13-4e6f-b636-9949b5e9f238-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz\" (UID: \"92472797-0d13-4e6f-b636-9949b5e9f238\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:43:43.617567 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.617472 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbqbq\" (UniqueName: \"kubernetes.io/projected/558c0c9a-d249-4941-9ec6-ef7df138048c-kube-api-access-wbqbq\") pod \"payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td\" (UID: \"558c0c9a-d249-4941-9ec6-ef7df138048c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" Jul 30 01:43:43.617567 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.617523 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/558c0c9a-d249-4941-9ec6-ef7df138048c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td\" (UID: \"558c0c9a-d249-4941-9ec6-ef7df138048c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" Jul 30 01:43:43.618335 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.618316 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/558c0c9a-d249-4941-9ec6-ef7df138048c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td\" (UID: \"558c0c9a-d249-4941-9ec6-ef7df138048c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" Jul 30 01:43:43.625634 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.625608 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbqbq\" (UniqueName: \"kubernetes.io/projected/558c0c9a-d249-4941-9ec6-ef7df138048c-kube-api-access-wbqbq\") pod \"payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td\" (UID: \"558c0c9a-d249-4941-9ec6-ef7df138048c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" Jul 30 01:43:43.773036 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.772996 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:43:43.802565 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.802537 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" Jul 30 01:43:43.927483 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.927456 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz"] Jul 30 01:43:43.930127 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:43:43.930091 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92472797_0d13_4e6f_b636_9949b5e9f238.slice/crio-5df8b7b04db3c3eb1e96c6a351fd01b40f7ce1567a9a1c81dfb9f3e3f95babac WatchSource:0}: Error finding container 5df8b7b04db3c3eb1e96c6a351fd01b40f7ce1567a9a1c81dfb9f3e3f95babac: Status 404 returned error can't find the container with id 5df8b7b04db3c3eb1e96c6a351fd01b40f7ce1567a9a1c81dfb9f3e3f95babac Jul 30 01:43:43.931916 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.931885 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 01:43:43.953797 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:43.953767 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td"] Jul 30 01:43:43.955812 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:43:43.955787 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod558c0c9a_d249_4941_9ec6_ef7df138048c.slice/crio-bbcd099d7bd2ac952e4befe59ea29212ece5304cce4b2aa9948ff756c1094e4f WatchSource:0}: Error finding container bbcd099d7bd2ac952e4befe59ea29212ece5304cce4b2aa9948ff756c1094e4f: Status 404 returned error can't find the container with id bbcd099d7bd2ac952e4befe59ea29212ece5304cce4b2aa9948ff756c1094e4f Jul 30 01:43:44.943261 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:44.943223 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" event={"ID":"92472797-0d13-4e6f-b636-9949b5e9f238","Type":"ContainerStarted","Data":"5df8b7b04db3c3eb1e96c6a351fd01b40f7ce1567a9a1c81dfb9f3e3f95babac"} Jul 30 01:43:44.946028 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:44.945966 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" event={"ID":"558c0c9a-d249-4941-9ec6-ef7df138048c","Type":"ContainerStarted","Data":"bbcd099d7bd2ac952e4befe59ea29212ece5304cce4b2aa9948ff756c1094e4f"} Jul 30 01:43:46.960455 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:46.960413 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" event={"ID":"558c0c9a-d249-4941-9ec6-ef7df138048c","Type":"ContainerStarted","Data":"f225143ee20e039c7010faf1ee0f6c76951d99233ac33f016d7909e8abeb9237"} Jul 30 01:43:46.960877 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:46.960577 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" Jul 30 01:43:46.962073 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:46.962051 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" event={"ID":"92472797-0d13-4e6f-b636-9949b5e9f238","Type":"ContainerStarted","Data":"a5eda8d8951bd0a20564f9e3f6370a55bc1d78ae46a5c55aa1c103ee82ef7ed2"} Jul 30 01:43:46.962203 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:46.962176 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:43:46.975383 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:46.975315 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" podStartSLOduration=1.376359817 podStartE2EDuration="3.975295574s" totalImagesPullingTime="2.598935757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:43:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:43:46.97492722 +0000 UTC m=+3629.775884323" watchObservedRunningTime="2026-07-30 01:43:46.975295574 +0000 UTC m=+3629.776252681" Jul 30 01:43:46.994169 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:46.994107 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" podStartSLOduration=2.369187443 podStartE2EDuration="4.994090561s" totalImagesPullingTime="2.624903118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:43:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:43:46.991804478 +0000 UTC m=+3629.792761588" watchObservedRunningTime="2026-07-30 01:43:46.994090561 +0000 UTC m=+3629.795047667" Jul 30 01:43:52.973165 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:52.973133 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:43:57.970224 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:43:57.970193 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" Jul 30 01:44:06.742218 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:44:06.742177 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:44:26.747112 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:44:26.747071 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:00.144461 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.144408 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-09b0d2-29756265-hjgqg"] Jul 30 01:45:00.148084 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.148063 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-55b321-29756265-tmcjz"] Jul 30 01:45:00.148222 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.148196 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-09b0d2-29756265-hjgqg" Jul 30 01:45:00.150663 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.150642 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\"" Jul 30 01:45:00.151175 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.151160 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756265-kwh67"] Jul 30 01:45:00.151305 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.151286 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-55b321-29756265-tmcjz" Jul 30 01:45:00.155026 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.155003 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756265-kwh67" Jul 30 01:45:00.155379 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.154888 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-b64c39-29756265-9cks8"] Jul 30 01:45:00.162394 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.162362 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fba93c-29756265-dh7xv"] Jul 30 01:45:00.163558 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.163538 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-b64c39-29756265-9cks8" Jul 30 01:45:00.167100 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.167074 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-09b0d2-29756265-hjgqg" Jul 30 01:45:00.169725 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.169701 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-09b0d2-29756265-hjgqg"] Jul 30 01:45:00.169945 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.169813 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fba93c-29756265-dh7xv" Jul 30 01:45:00.170085 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.170069 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756265-kwh67"] Jul 30 01:45:00.170202 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.170184 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-b64c39-29756265-9cks8"] Jul 30 01:45:00.171290 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.171268 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-55b321-29756265-tmcjz"] Jul 30 01:45:00.175180 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.175119 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fba93c-29756265-dh7xv"] Jul 30 01:45:00.182427 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.182399 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-55b321-29756265-tmcjz" Jul 30 01:45:00.218802 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.218773 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756265-kwh67" Jul 30 01:45:00.229812 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.229760 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-b64c39-29756265-9cks8" Jul 30 01:45:00.273603 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.273393 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fba93c-29756265-dh7xv" Jul 30 01:45:00.410149 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.410107 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-55b321-29756265-tmcjz"] Jul 30 01:45:00.411743 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:45:00.411688 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b1f36c3_1e52_4b03_bc75_7643ae87d34c.slice/crio-dd325b63b308fc3a4455037e313b2cefc50005b8598e314a86dcd398bb62e80b WatchSource:0}: Error finding container dd325b63b308fc3a4455037e313b2cefc50005b8598e314a86dcd398bb62e80b: Status 404 returned error can't find the container with id dd325b63b308fc3a4455037e313b2cefc50005b8598e314a86dcd398bb62e80b Jul 30 01:45:00.438799 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.438747 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756265-kwh67"] Jul 30 01:45:00.446508 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:45:00.446469 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeac78ae6_5093_4547_8010_d2c981284648.slice/crio-889c0ee81c304d5b5a419c5733e5c25dc53017691559efa3825d18ee8cb8dc67 WatchSource:0}: Error finding container 889c0ee81c304d5b5a419c5733e5c25dc53017691559efa3825d18ee8cb8dc67: Status 404 returned error can't find the container with id 889c0ee81c304d5b5a419c5733e5c25dc53017691559efa3825d18ee8cb8dc67 Jul 30 01:45:00.468673 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.468644 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-b64c39-29756265-9cks8"] Jul 30 01:45:00.469876 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:45:00.469841 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcad0cf5c_b02f_4164_b732_888374852831.slice/crio-1898a46a52258bc0a0c5eb6dd6c8338f02fc6d62b398c8c17fd8feb97dd95172 WatchSource:0}: Error finding container 1898a46a52258bc0a0c5eb6dd6c8338f02fc6d62b398c8c17fd8feb97dd95172: Status 404 returned error can't find the container with id 1898a46a52258bc0a0c5eb6dd6c8338f02fc6d62b398c8c17fd8feb97dd95172 Jul 30 01:45:00.495686 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.495662 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fba93c-29756265-dh7xv"] Jul 30 01:45:00.500337 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:45:00.500304 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7da35555_8dc2_47be_af2d_2f61f5df3b1c.slice/crio-af8d29c2d6264b71683a79502d31bdce31c0eeb167c9d7045d08f7547d83d89b WatchSource:0}: Error finding container af8d29c2d6264b71683a79502d31bdce31c0eeb167c9d7045d08f7547d83d89b: Status 404 returned error can't find the container with id af8d29c2d6264b71683a79502d31bdce31c0eeb167c9d7045d08f7547d83d89b Jul 30 01:45:00.566959 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:00.566832 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-09b0d2-29756265-hjgqg"] Jul 30 01:45:01.383996 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:01.383954 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fba93c-29756265-dh7xv" event={"ID":"7da35555-8dc2-47be-af2d-2f61f5df3b1c","Type":"ContainerStarted","Data":"ca21c92a3519d8f98f2442e5990e824462ccb7c9d5eed6d3286b8e04ed81ae44"} Jul 30 01:45:01.384494 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:01.384020 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fba93c-29756265-dh7xv" event={"ID":"7da35555-8dc2-47be-af2d-2f61f5df3b1c","Type":"ContainerStarted","Data":"af8d29c2d6264b71683a79502d31bdce31c0eeb167c9d7045d08f7547d83d89b"} Jul 30 01:45:01.385585 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:01.385559 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-55b321-29756265-tmcjz" event={"ID":"9b1f36c3-1e52-4b03-bc75-7643ae87d34c","Type":"ContainerStarted","Data":"949e68656f502b7b06b4f01c2efc53667d3466e9ec07cb24044a9cedbae96e4e"} Jul 30 01:45:01.385702 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:01.385610 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-55b321-29756265-tmcjz" event={"ID":"9b1f36c3-1e52-4b03-bc75-7643ae87d34c","Type":"ContainerStarted","Data":"dd325b63b308fc3a4455037e313b2cefc50005b8598e314a86dcd398bb62e80b"} Jul 30 01:45:01.387197 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:01.387132 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-09b0d2-29756265-hjgqg" event={"ID":"da0d2adb-62ef-4506-b382-39eee9a084be","Type":"ContainerStarted","Data":"bdbaacaf3081bdd798be2d0abae2afd44048ce8dd526434bedcb037839af3b70"} Jul 30 01:45:01.387197 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:01.387181 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-09b0d2-29756265-hjgqg" event={"ID":"da0d2adb-62ef-4506-b382-39eee9a084be","Type":"ContainerStarted","Data":"603345b1e0d1d38c753b6bb436c6bfafc26f407536748b3f03ec2e8ee7dd04fb"} Jul 30 01:45:01.388684 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:01.388665 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-b64c39-29756265-9cks8" event={"ID":"cad0cf5c-b02f-4164-b732-888374852831","Type":"ContainerStarted","Data":"8f3d5f3a3bdd94735a8abd560e348b789548969509ae51a2fdc8c46e5176f0ce"} Jul 30 01:45:01.388684 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:01.388708 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-b64c39-29756265-9cks8" event={"ID":"cad0cf5c-b02f-4164-b732-888374852831","Type":"ContainerStarted","Data":"1898a46a52258bc0a0c5eb6dd6c8338f02fc6d62b398c8c17fd8feb97dd95172"} Jul 30 01:45:01.390090 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:01.390065 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756265-kwh67" event={"ID":"eac78ae6-5093-4547-8010-d2c981284648","Type":"ContainerStarted","Data":"561f7cc6ea44b21e0f0e7736ef208969a716d505364f399f0fa65bcbee514ead"} Jul 30 01:45:01.390177 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:01.390110 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756265-kwh67" event={"ID":"eac78ae6-5093-4547-8010-d2c981284648","Type":"ContainerStarted","Data":"889c0ee81c304d5b5a419c5733e5c25dc53017691559efa3825d18ee8cb8dc67"} Jul 30 01:45:01.403057 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:01.402991 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fba93c-29756265-dh7xv" podStartSLOduration=1.402967163 podStartE2EDuration="1.402967163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:45:01.397854303 +0000 UTC m=+3704.198811410" watchObservedRunningTime="2026-07-30 01:45:01.402967163 +0000 UTC m=+3704.203924270" Jul 30 01:45:01.413384 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:01.413324 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756265-kwh67" podStartSLOduration=1.413308052 podStartE2EDuration="1.413308052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:45:01.410946694 +0000 UTC m=+3704.211903801" watchObservedRunningTime="2026-07-30 01:45:01.413308052 +0000 UTC m=+3704.214265157" Jul 30 01:45:01.429185 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:01.429109 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-55b321-29756265-tmcjz" podStartSLOduration=1.42909053 podStartE2EDuration="1.42909053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:45:01.426622114 +0000 UTC m=+3704.227579221" watchObservedRunningTime="2026-07-30 01:45:01.42909053 +0000 UTC m=+3704.230047636" Jul 30 01:45:01.443748 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:01.443691 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-b64c39-29756265-9cks8" podStartSLOduration=1.443673671 podStartE2EDuration="1.443673671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:45:01.441221546 +0000 UTC m=+3704.242178653" watchObservedRunningTime="2026-07-30 01:45:01.443673671 +0000 UTC m=+3704.244630848" Jul 30 01:45:01.454506 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:01.454442 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-09b0d2-29756265-hjgqg" podStartSLOduration=1.454424719 podStartE2EDuration="1.454424719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:45:01.4540133 +0000 UTC m=+3704.254970407" watchObservedRunningTime="2026-07-30 01:45:01.454424719 +0000 UTC m=+3704.255381825" Jul 30 01:45:02.398130 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:02.398097 2564 generic.go:356] "Generic (PLEG): container finished" podID="7da35555-8dc2-47be-af2d-2f61f5df3b1c" containerID="ca21c92a3519d8f98f2442e5990e824462ccb7c9d5eed6d3286b8e04ed81ae44" exitCode=0 Jul 30 01:45:02.398567 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:02.398167 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fba93c-29756265-dh7xv" event={"ID":"7da35555-8dc2-47be-af2d-2f61f5df3b1c","Type":"ContainerDied","Data":"ca21c92a3519d8f98f2442e5990e824462ccb7c9d5eed6d3286b8e04ed81ae44"} Jul 30 01:45:02.399731 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:02.399706 2564 generic.go:356] "Generic (PLEG): container finished" podID="9b1f36c3-1e52-4b03-bc75-7643ae87d34c" containerID="949e68656f502b7b06b4f01c2efc53667d3466e9ec07cb24044a9cedbae96e4e" exitCode=0 Jul 30 01:45:02.399819 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:02.399774 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-55b321-29756265-tmcjz" event={"ID":"9b1f36c3-1e52-4b03-bc75-7643ae87d34c","Type":"ContainerDied","Data":"949e68656f502b7b06b4f01c2efc53667d3466e9ec07cb24044a9cedbae96e4e"} Jul 30 01:45:02.401305 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:02.401274 2564 generic.go:356] "Generic (PLEG): container finished" podID="da0d2adb-62ef-4506-b382-39eee9a084be" containerID="bdbaacaf3081bdd798be2d0abae2afd44048ce8dd526434bedcb037839af3b70" exitCode=0 Jul 30 01:45:02.401439 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:02.401317 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-09b0d2-29756265-hjgqg" event={"ID":"da0d2adb-62ef-4506-b382-39eee9a084be","Type":"ContainerDied","Data":"bdbaacaf3081bdd798be2d0abae2afd44048ce8dd526434bedcb037839af3b70"} Jul 30 01:45:02.402775 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:02.402749 2564 generic.go:356] "Generic (PLEG): container finished" podID="cad0cf5c-b02f-4164-b732-888374852831" containerID="8f3d5f3a3bdd94735a8abd560e348b789548969509ae51a2fdc8c46e5176f0ce" exitCode=0 Jul 30 01:45:02.402872 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:02.402836 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-b64c39-29756265-9cks8" event={"ID":"cad0cf5c-b02f-4164-b732-888374852831","Type":"ContainerDied","Data":"8f3d5f3a3bdd94735a8abd560e348b789548969509ae51a2fdc8c46e5176f0ce"} Jul 30 01:45:02.404251 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:02.404232 2564 generic.go:356] "Generic (PLEG): container finished" podID="eac78ae6-5093-4547-8010-d2c981284648" containerID="561f7cc6ea44b21e0f0e7736ef208969a716d505364f399f0fa65bcbee514ead" exitCode=0 Jul 30 01:45:02.404347 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:02.404312 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756265-kwh67" event={"ID":"eac78ae6-5093-4547-8010-d2c981284648","Type":"ContainerDied","Data":"561f7cc6ea44b21e0f0e7736ef208969a716d505364f399f0fa65bcbee514ead"} Jul 30 01:45:03.575313 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:03.575279 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-b64c39-29756265-9cks8" Jul 30 01:45:03.655410 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:03.655379 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fba93c-29756265-dh7xv" Jul 30 01:45:03.659568 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:03.659544 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-09b0d2-29756265-hjgqg" Jul 30 01:45:03.663102 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:03.663031 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-55b321-29756265-tmcjz" Jul 30 01:45:03.666698 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:03.666675 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756265-kwh67" Jul 30 01:45:04.418290 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:04.418262 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fba93c-29756265-dh7xv" Jul 30 01:45:04.418489 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:04.418258 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fba93c-29756265-dh7xv" event={"ID":"7da35555-8dc2-47be-af2d-2f61f5df3b1c","Type":"ContainerDied","Data":"af8d29c2d6264b71683a79502d31bdce31c0eeb167c9d7045d08f7547d83d89b"} Jul 30 01:45:04.418489 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:04.418406 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af8d29c2d6264b71683a79502d31bdce31c0eeb167c9d7045d08f7547d83d89b" Jul 30 01:45:04.419657 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:04.419636 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-55b321-29756265-tmcjz" Jul 30 01:45:04.419794 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:04.419668 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-55b321-29756265-tmcjz" event={"ID":"9b1f36c3-1e52-4b03-bc75-7643ae87d34c","Type":"ContainerDied","Data":"dd325b63b308fc3a4455037e313b2cefc50005b8598e314a86dcd398bb62e80b"} Jul 30 01:45:04.419794 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:04.419723 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd325b63b308fc3a4455037e313b2cefc50005b8598e314a86dcd398bb62e80b" Jul 30 01:45:04.421164 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:04.421146 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-09b0d2-29756265-hjgqg" Jul 30 01:45:04.421285 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:04.421185 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-09b0d2-29756265-hjgqg" event={"ID":"da0d2adb-62ef-4506-b382-39eee9a084be","Type":"ContainerDied","Data":"603345b1e0d1d38c753b6bb436c6bfafc26f407536748b3f03ec2e8ee7dd04fb"} Jul 30 01:45:04.421285 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:04.421224 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="603345b1e0d1d38c753b6bb436c6bfafc26f407536748b3f03ec2e8ee7dd04fb" Jul 30 01:45:04.422739 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:04.422719 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-b64c39-29756265-9cks8" Jul 30 01:45:04.422834 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:04.422711 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-b64c39-29756265-9cks8" event={"ID":"cad0cf5c-b02f-4164-b732-888374852831","Type":"ContainerDied","Data":"1898a46a52258bc0a0c5eb6dd6c8338f02fc6d62b398c8c17fd8feb97dd95172"} Jul 30 01:45:04.422907 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:04.422844 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1898a46a52258bc0a0c5eb6dd6c8338f02fc6d62b398c8c17fd8feb97dd95172" Jul 30 01:45:04.424074 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:04.424058 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756265-kwh67" Jul 30 01:45:04.424177 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:04.424118 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756265-kwh67" event={"ID":"eac78ae6-5093-4547-8010-d2c981284648","Type":"ContainerDied","Data":"889c0ee81c304d5b5a419c5733e5c25dc53017691559efa3825d18ee8cb8dc67"} Jul 30 01:45:04.424177 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:04.424158 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="889c0ee81c304d5b5a419c5733e5c25dc53017691559efa3825d18ee8cb8dc67" Jul 30 01:45:21.673867 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.673831 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:21.964756 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.964672 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq"] Jul 30 01:45:21.965236 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.965209 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eac78ae6-5093-4547-8010-d2c981284648" containerName="cleanup" Jul 30 01:45:21.965236 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.965234 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eac78ae6-5093-4547-8010-d2c981284648" containerName="cleanup" Jul 30 01:45:21.965398 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.965250 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cad0cf5c-b02f-4164-b732-888374852831" containerName="cleanup" Jul 30 01:45:21.965398 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.965259 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cad0cf5c-b02f-4164-b732-888374852831" containerName="cleanup" Jul 30 01:45:21.965398 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.965278 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b1f36c3-1e52-4b03-bc75-7643ae87d34c" containerName="cleanup" Jul 30 01:45:21.965398 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.965284 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b1f36c3-1e52-4b03-bc75-7643ae87d34c" containerName="cleanup" Jul 30 01:45:21.965398 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.965292 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da0d2adb-62ef-4506-b382-39eee9a084be" containerName="cleanup" Jul 30 01:45:21.965398 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.965297 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="da0d2adb-62ef-4506-b382-39eee9a084be" containerName="cleanup" Jul 30 01:45:21.965398 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.965310 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7da35555-8dc2-47be-af2d-2f61f5df3b1c" containerName="cleanup" Jul 30 01:45:21.965398 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.965315 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7da35555-8dc2-47be-af2d-2f61f5df3b1c" containerName="cleanup" Jul 30 01:45:21.965398 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.965393 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7da35555-8dc2-47be-af2d-2f61f5df3b1c" containerName="cleanup" Jul 30 01:45:21.965398 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.965402 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eac78ae6-5093-4547-8010-d2c981284648" containerName="cleanup" Jul 30 01:45:21.965689 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.965408 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b1f36c3-1e52-4b03-bc75-7643ae87d34c" containerName="cleanup" Jul 30 01:45:21.965689 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.965414 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cad0cf5c-b02f-4164-b732-888374852831" containerName="cleanup" Jul 30 01:45:21.965689 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.965421 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="da0d2adb-62ef-4506-b382-39eee9a084be" containerName="cleanup" Jul 30 01:45:21.968642 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.968625 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:21.971202 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.971183 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-06c66b-openshift-default-dockercfg-hdvt2\"" Jul 30 01:45:21.982992 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.982957 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq"] Jul 30 01:45:21.990063 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:21.990029 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:22.097634 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.097604 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:22.119874 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.119842 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-envoy\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.120074 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.119883 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-workload-socket\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.120074 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.119976 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-credential-socket\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.120074 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.120004 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcsnj\" (UniqueName: \"kubernetes.io/projected/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-kube-api-access-dcsnj\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.120074 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.120046 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-workload-certs\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.120226 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.120120 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istiod-ca-cert\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.120226 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.120146 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-data\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.120226 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.120165 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-token\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.120226 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.120186 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-podinfo\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.220772 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.220683 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-workload-socket\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.220772 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.220722 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-credential-socket\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.220772 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.220745 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcsnj\" (UniqueName: \"kubernetes.io/projected/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-kube-api-access-dcsnj\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.221164 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.220790 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-workload-certs\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.221164 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.220959 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istiod-ca-cert\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.221164 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.221004 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-data\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.221164 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.221031 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-token\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.221164 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.221062 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-podinfo\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.221164 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.221132 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-envoy\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.221497 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.221213 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-workload-socket\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.221497 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.221269 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-credential-socket\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.221497 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.221396 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-data\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.221497 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.221436 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-workload-certs\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.221701 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.221679 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istiod-ca-cert\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.223494 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.223472 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-envoy\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.223645 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.223623 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-podinfo\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.228353 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.228327 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-token\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.228432 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.228419 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcsnj\" (UniqueName: \"kubernetes.io/projected/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-kube-api-access-dcsnj\") pod \"e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.279251 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.279206 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:22.429535 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.429503 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq"] Jul 30 01:45:22.430636 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:45:22.430613 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb0b8046_a1c9_4494_b4d9_8b87bb2ef6ee.slice/crio-8feedf7b4e3ee1adf7658b07e3f9034e738ecf3bc909f2c289ae87291a950862 WatchSource:0}: Error finding container 8feedf7b4e3ee1adf7658b07e3f9034e738ecf3bc909f2c289ae87291a950862: Status 404 returned error can't find the container with id 8feedf7b4e3ee1adf7658b07e3f9034e738ecf3bc909f2c289ae87291a950862 Jul 30 01:45:22.432784 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.432753 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:45:22.432878 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.432820 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:45:22.432878 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.432848 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:45:22.534385 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.534351 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" event={"ID":"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee","Type":"ContainerStarted","Data":"08c871def0f81e0785ae0beeea7bded8e8cbcc0c1a5b688c3f4e8fb7667cdfba"} Jul 30 01:45:22.534553 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.534404 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" event={"ID":"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee","Type":"ContainerStarted","Data":"8feedf7b4e3ee1adf7658b07e3f9034e738ecf3bc909f2c289ae87291a950862"} Jul 30 01:45:22.553718 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:22.553667 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" podStartSLOduration=1.553649308 podStartE2EDuration="1.553649308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:45:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:45:22.551391446 +0000 UTC m=+3725.352348555" watchObservedRunningTime="2026-07-30 01:45:22.553649308 +0000 UTC m=+3725.354606413" Jul 30 01:45:23.280279 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:23.280245 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:23.285238 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:23.285213 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:23.540290 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:23.540208 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:23.541269 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:23.541250 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:24.453531 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:24.453494 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:24.553984 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:24.553948 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:25.542698 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:25.542666 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8"] Jul 30 01:45:25.546463 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:25.546445 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:25.548054 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:25.548026 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdl8v\" (UniqueName: \"kubernetes.io/projected/0cf0765e-2f30-4cb3-83e3-9d322281bf04-kube-api-access-sdl8v\") pod \"maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8\" (UID: \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:25.548196 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:25.548093 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf0765e-2f30-4cb3-83e3-9d322281bf04-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8\" (UID: \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:25.548196 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:25.548144 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cf0765e-2f30-4cb3-83e3-9d322281bf04-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8\" (UID: \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:25.548720 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:25.548701 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-06c66b-serving-cert\"" Jul 30 01:45:25.555363 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:25.555335 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8"] Jul 30 01:45:25.572948 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:25.572883 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:25.649603 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:25.649565 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cf0765e-2f30-4cb3-83e3-9d322281bf04-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8\" (UID: \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:25.649805 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:25.649702 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdl8v\" (UniqueName: \"kubernetes.io/projected/0cf0765e-2f30-4cb3-83e3-9d322281bf04-kube-api-access-sdl8v\") pod \"maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8\" (UID: \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:25.649805 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:25.649764 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf0765e-2f30-4cb3-83e3-9d322281bf04-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8\" (UID: \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:25.649805 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:45:25.649705 2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-serving-cert: secret "maas-api-e2e-ipp-cleanup-06c66b-serving-cert" not found Jul 30 01:45:25.650002 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:45:25.649882 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf0765e-2f30-4cb3-83e3-9d322281bf04-maas-api-tls podName:0cf0765e-2f30-4cb3-83e3-9d322281bf04 nodeName:}" failed. No retries permitted until 2026-07-30 01:45:26.149854023 +0000 UTC m=+3728.950811118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0cf0765e-2f30-4cb3-83e3-9d322281bf04-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" (UID: "0cf0765e-2f30-4cb3-83e3-9d322281bf04") : secret "maas-api-e2e-ipp-cleanup-06c66b-serving-cert" not found Jul 30 01:45:25.650618 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:25.650594 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf0765e-2f30-4cb3-83e3-9d322281bf04-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8\" (UID: \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:25.658777 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:25.658753 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdl8v\" (UniqueName: \"kubernetes.io/projected/0cf0765e-2f30-4cb3-83e3-9d322281bf04-kube-api-access-sdl8v\") pod \"maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8\" (UID: \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:25.815134 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:25.815043 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:26.153885 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.153846 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cf0765e-2f30-4cb3-83e3-9d322281bf04-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8\" (UID: \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:26.156347 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.156322 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cf0765e-2f30-4cb3-83e3-9d322281bf04-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8\" (UID: \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:26.159584 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.159565 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:26.160134 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.160109 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:26.263367 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.263334 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:26.274206 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.274170 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv"] Jul 30 01:45:26.279565 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.279537 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" Jul 30 01:45:26.282604 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.282578 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-06c66b\"" Jul 30 01:45:26.282723 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.282579 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-06c66b-dockercfg-72ldr\"" Jul 30 01:45:26.292085 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.292059 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv"] Jul 30 01:45:26.313062 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:45:26.313039 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf0765e_2f30_4cb3_83e3_9d322281bf04.slice/crio-62dd218733c0e06f135bb2b6992ec9f2ecb5427c4e59e9168ca719cf261ea090 WatchSource:0}: Error finding container 62dd218733c0e06f135bb2b6992ec9f2ecb5427c4e59e9168ca719cf261ea090: Status 404 returned error can't find the container with id 62dd218733c0e06f135bb2b6992ec9f2ecb5427c4e59e9168ca719cf261ea090 Jul 30 01:45:26.313772 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.313752 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8"] Jul 30 01:45:26.318434 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.318411 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8"] Jul 30 01:45:26.322148 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.322128 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" Jul 30 01:45:26.331691 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.331668 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8"] Jul 30 01:45:26.355793 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.355757 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f87gb\" (UniqueName: \"kubernetes.io/projected/cae76037-9035-4b57-aaae-fa818a2dc7c2-kube-api-access-f87gb\") pod \"payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv\" (UID: \"cae76037-9035-4b57-aaae-fa818a2dc7c2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" Jul 30 01:45:26.355793 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.355794 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btplg\" (UniqueName: \"kubernetes.io/projected/9ad201f7-0caa-4e24-aef2-043f66981419-kube-api-access-btplg\") pod \"payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8\" (UID: \"9ad201f7-0caa-4e24-aef2-043f66981419\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" Jul 30 01:45:26.356088 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.355871 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cae76037-9035-4b57-aaae-fa818a2dc7c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv\" (UID: \"cae76037-9035-4b57-aaae-fa818a2dc7c2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" Jul 30 01:45:26.356088 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.355973 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ad201f7-0caa-4e24-aef2-043f66981419-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8\" (UID: \"9ad201f7-0caa-4e24-aef2-043f66981419\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" Jul 30 01:45:26.456672 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.456573 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f87gb\" (UniqueName: \"kubernetes.io/projected/cae76037-9035-4b57-aaae-fa818a2dc7c2-kube-api-access-f87gb\") pod \"payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv\" (UID: \"cae76037-9035-4b57-aaae-fa818a2dc7c2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" Jul 30 01:45:26.456672 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.456633 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btplg\" (UniqueName: \"kubernetes.io/projected/9ad201f7-0caa-4e24-aef2-043f66981419-kube-api-access-btplg\") pod \"payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8\" (UID: \"9ad201f7-0caa-4e24-aef2-043f66981419\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" Jul 30 01:45:26.456932 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.456701 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cae76037-9035-4b57-aaae-fa818a2dc7c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv\" (UID: \"cae76037-9035-4b57-aaae-fa818a2dc7c2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" Jul 30 01:45:26.456932 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.456800 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ad201f7-0caa-4e24-aef2-043f66981419-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8\" (UID: \"9ad201f7-0caa-4e24-aef2-043f66981419\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" Jul 30 01:45:26.457485 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.457464 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cae76037-9035-4b57-aaae-fa818a2dc7c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv\" (UID: \"cae76037-9035-4b57-aaae-fa818a2dc7c2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" Jul 30 01:45:26.457551 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.457531 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ad201f7-0caa-4e24-aef2-043f66981419-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8\" (UID: \"9ad201f7-0caa-4e24-aef2-043f66981419\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" Jul 30 01:45:26.466636 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.466609 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f87gb\" (UniqueName: \"kubernetes.io/projected/cae76037-9035-4b57-aaae-fa818a2dc7c2-kube-api-access-f87gb\") pod \"payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv\" (UID: \"cae76037-9035-4b57-aaae-fa818a2dc7c2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" Jul 30 01:45:26.467293 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.467267 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btplg\" (UniqueName: \"kubernetes.io/projected/9ad201f7-0caa-4e24-aef2-043f66981419-kube-api-access-btplg\") pod \"payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8\" (UID: \"9ad201f7-0caa-4e24-aef2-043f66981419\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" Jul 30 01:45:26.560335 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.560298 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" event={"ID":"0cf0765e-2f30-4cb3-83e3-9d322281bf04","Type":"ContainerStarted","Data":"62dd218733c0e06f135bb2b6992ec9f2ecb5427c4e59e9168ca719cf261ea090"} Jul 30 01:45:26.593632 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.593592 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" Jul 30 01:45:26.634951 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.634889 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" Jul 30 01:45:26.738789 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.738747 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv"] Jul 30 01:45:26.740092 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:45:26.740063 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae76037_9035_4b57_aaae_fa818a2dc7c2.slice/crio-184e7858e40770610730710f7e5decd91440e1aed07938b92e41c1fb06830dcf WatchSource:0}: Error finding container 184e7858e40770610730710f7e5decd91440e1aed07938b92e41c1fb06830dcf: Status 404 returned error can't find the container with id 184e7858e40770610730710f7e5decd91440e1aed07938b92e41c1fb06830dcf Jul 30 01:45:26.785638 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:26.785609 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8"] Jul 30 01:45:26.788806 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:45:26.788781 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ad201f7_0caa_4e24_aef2_043f66981419.slice/crio-206a013fc12956cb50d44517250e5f5f4ef8bc991087b3ac3f7f3fb26237a656 WatchSource:0}: Error finding container 206a013fc12956cb50d44517250e5f5f4ef8bc991087b3ac3f7f3fb26237a656: Status 404 returned error can't find the container with id 206a013fc12956cb50d44517250e5f5f4ef8bc991087b3ac3f7f3fb26237a656 Jul 30 01:45:27.567688 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:27.567653 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" event={"ID":"0cf0765e-2f30-4cb3-83e3-9d322281bf04","Type":"ContainerStarted","Data":"084daf29193e14664e5294df25733147bbb194b11a6c5bbacc149712e89bf793"} Jul 30 01:45:27.568204 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:27.567762 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:27.569300 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:27.569275 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" event={"ID":"9ad201f7-0caa-4e24-aef2-043f66981419","Type":"ContainerStarted","Data":"4c290616d82f1564f6b772c2f317d1e3c48417db01ecd1256a20cc3268a2663d"} Jul 30 01:45:27.569423 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:27.569318 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" event={"ID":"9ad201f7-0caa-4e24-aef2-043f66981419","Type":"ContainerStarted","Data":"206a013fc12956cb50d44517250e5f5f4ef8bc991087b3ac3f7f3fb26237a656"} Jul 30 01:45:27.569423 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:27.569349 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" Jul 30 01:45:27.570830 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:27.570805 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" event={"ID":"cae76037-9035-4b57-aaae-fa818a2dc7c2","Type":"ContainerStarted","Data":"7ab8d7b0873c531776f22ffca5ad9120b68ebcaeec3aed290f364a6cf5559abe"} Jul 30 01:45:27.570959 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:27.570843 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" event={"ID":"cae76037-9035-4b57-aaae-fa818a2dc7c2","Type":"ContainerStarted","Data":"184e7858e40770610730710f7e5decd91440e1aed07938b92e41c1fb06830dcf"} Jul 30 01:45:27.571044 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:27.570992 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" Jul 30 01:45:27.588210 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:27.588153 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" podStartSLOduration=2.111452598 podStartE2EDuration="2.5881363s" totalImagesPullingTime="476.683702ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:45:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:45:27.585831795 +0000 UTC m=+3730.386788894" watchObservedRunningTime="2026-07-30 01:45:27.5881363 +0000 UTC m=+3730.389093433" Jul 30 01:45:27.603046 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:27.602988 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" podStartSLOduration=1.602968046 podStartE2EDuration="1.602968046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:45:27.601485201 +0000 UTC m=+3730.402442307" watchObservedRunningTime="2026-07-30 01:45:27.602968046 +0000 UTC m=+3730.403925160" Jul 30 01:45:27.618209 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:27.618158 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" podStartSLOduration=1.618142937 podStartE2EDuration="1.618142937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:45:27.616319094 +0000 UTC m=+3730.417276210" watchObservedRunningTime="2026-07-30 01:45:27.618142937 +0000 UTC m=+3730.419100043" Jul 30 01:45:33.581871 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:33.581841 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:38.541752 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:38.541657 2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 30 01:45:38.541752 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:38.541724 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" podUID="bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 30 01:45:38.578443 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:38.578414 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" Jul 30 01:45:38.578628 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:38.578491 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" Jul 30 01:45:38.648237 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:38.648155 2564 generic.go:356] "Generic (PLEG): container finished" podID="bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" containerID="08c871def0f81e0785ae0beeea7bded8e8cbcc0c1a5b688c3f4e8fb7667cdfba" exitCode=0 Jul 30 01:45:38.648406 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:38.648228 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" event={"ID":"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee","Type":"ContainerDied","Data":"08c871def0f81e0785ae0beeea7bded8e8cbcc0c1a5b688c3f4e8fb7667cdfba"} Jul 30 01:45:38.648703 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:38.648686 2564 scope.go:117] "RemoveContainer" containerID="08c871def0f81e0785ae0beeea7bded8e8cbcc0c1a5b688c3f4e8fb7667cdfba" Jul 30 01:45:38.649851 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:38.649822 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:45:38.649952 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:38.649934 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:45:38.650001 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:38.649987 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 01:45:39.656393 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:39.656362 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" event={"ID":"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee","Type":"ContainerStarted","Data":"c177201cb95f158626712acaddd0285673903140c8e6b6da9ca29fba2efb3a30"} Jul 30 01:45:40.279356 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:40.279308 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:40.279654 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:40.279631 2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 30 01:45:40.279713 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:40.279693 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" podUID="bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 30 01:45:41.279684 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:41.279650 2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 30 01:45:41.280091 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:41.279700 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" podUID="bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 30 01:45:42.280143 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:42.280108 2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 30 01:45:42.280554 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:42.280169 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" podUID="bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 30 01:45:43.280165 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:43.280123 2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 30 01:45:43.280585 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:43.280198 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" podUID="bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 30 01:45:44.283944 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:44.283909 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:44.284390 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:44.284183 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:44.285094 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:44.285079 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:45:47.047709 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:47.047670 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:47.098155 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:47.098126 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:49.474612 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:49.474578 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:56.136787 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.136751 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8"] Jul 30 01:45:56.137281 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.136989 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" podUID="0cf0765e-2f30-4cb3-83e3-9d322281bf04" containerName="maas-api" containerID="cri-o://084daf29193e14664e5294df25733147bbb194b11a6c5bbacc149712e89bf793" gracePeriod=30 Jul 30 01:45:56.162273 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.162237 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:56.192572 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.192529 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8"] Jul 30 01:45:56.192850 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.192802 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" podUID="9ad201f7-0caa-4e24-aef2-043f66981419" containerName="payload-processing" containerID="cri-o://4c290616d82f1564f6b772c2f317d1e3c48417db01ecd1256a20cc3268a2663d" gracePeriod=30 Jul 30 01:45:56.200312 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.200281 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv"] Jul 30 01:45:56.201048 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.200652 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" podUID="cae76037-9035-4b57-aaae-fa818a2dc7c2" containerName="payload-pre-processing" containerID="cri-o://7ab8d7b0873c531776f22ffca5ad9120b68ebcaeec3aed290f364a6cf5559abe" gracePeriod=30 Jul 30 01:45:56.230503 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.230474 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:56.339808 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.339597 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:56.480846 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.480774 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:56.504143 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.504111 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" Jul 30 01:45:56.523416 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.523386 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" Jul 30 01:45:56.528833 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.528803 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:56.595202 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.595164 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f87gb\" (UniqueName: \"kubernetes.io/projected/cae76037-9035-4b57-aaae-fa818a2dc7c2-kube-api-access-f87gb\") pod \"cae76037-9035-4b57-aaae-fa818a2dc7c2\" (UID: \"cae76037-9035-4b57-aaae-fa818a2dc7c2\") " Jul 30 01:45:56.595372 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.595268 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cae76037-9035-4b57-aaae-fa818a2dc7c2-plugins-config-volume\") pod \"cae76037-9035-4b57-aaae-fa818a2dc7c2\" (UID: \"cae76037-9035-4b57-aaae-fa818a2dc7c2\") " Jul 30 01:45:56.595372 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.595296 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cf0765e-2f30-4cb3-83e3-9d322281bf04-maas-api-tls\") pod \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\" (UID: \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\") " Jul 30 01:45:56.595372 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.595311 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ad201f7-0caa-4e24-aef2-043f66981419-plugins-config-volume\") pod \"9ad201f7-0caa-4e24-aef2-043f66981419\" (UID: \"9ad201f7-0caa-4e24-aef2-043f66981419\") " Jul 30 01:45:56.595372 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.595330 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf0765e-2f30-4cb3-83e3-9d322281bf04-openshift-service-ca\") pod \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\" (UID: \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\") " Jul 30 01:45:56.595372 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.595350 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btplg\" (UniqueName: \"kubernetes.io/projected/9ad201f7-0caa-4e24-aef2-043f66981419-kube-api-access-btplg\") pod \"9ad201f7-0caa-4e24-aef2-043f66981419\" (UID: \"9ad201f7-0caa-4e24-aef2-043f66981419\") " Jul 30 01:45:56.595630 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.595401 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdl8v\" (UniqueName: \"kubernetes.io/projected/0cf0765e-2f30-4cb3-83e3-9d322281bf04-kube-api-access-sdl8v\") pod \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\" (UID: \"0cf0765e-2f30-4cb3-83e3-9d322281bf04\") " Jul 30 01:45:56.595765 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.595737 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cae76037-9035-4b57-aaae-fa818a2dc7c2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cae76037-9035-4b57-aaae-fa818a2dc7c2" (UID: "cae76037-9035-4b57-aaae-fa818a2dc7c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:45:56.595831 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.595742 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cf0765e-2f30-4cb3-83e3-9d322281bf04-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0cf0765e-2f30-4cb3-83e3-9d322281bf04" (UID: "0cf0765e-2f30-4cb3-83e3-9d322281bf04"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:45:56.595831 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.595743 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ad201f7-0caa-4e24-aef2-043f66981419-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9ad201f7-0caa-4e24-aef2-043f66981419" (UID: "9ad201f7-0caa-4e24-aef2-043f66981419"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:45:56.597537 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.597507 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cae76037-9035-4b57-aaae-fa818a2dc7c2-kube-api-access-f87gb" (OuterVolumeSpecName: "kube-api-access-f87gb") pod "cae76037-9035-4b57-aaae-fa818a2dc7c2" (UID: "cae76037-9035-4b57-aaae-fa818a2dc7c2"). InnerVolumeSpecName "kube-api-access-f87gb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:45:56.597636 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.597578 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cf0765e-2f30-4cb3-83e3-9d322281bf04-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0cf0765e-2f30-4cb3-83e3-9d322281bf04" (UID: "0cf0765e-2f30-4cb3-83e3-9d322281bf04"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 01:45:56.597707 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.597682 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cf0765e-2f30-4cb3-83e3-9d322281bf04-kube-api-access-sdl8v" (OuterVolumeSpecName: "kube-api-access-sdl8v") pod "0cf0765e-2f30-4cb3-83e3-9d322281bf04" (UID: "0cf0765e-2f30-4cb3-83e3-9d322281bf04"). InnerVolumeSpecName "kube-api-access-sdl8v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:45:56.597707 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.597681 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ad201f7-0caa-4e24-aef2-043f66981419-kube-api-access-btplg" (OuterVolumeSpecName: "kube-api-access-btplg") pod "9ad201f7-0caa-4e24-aef2-043f66981419" (UID: "9ad201f7-0caa-4e24-aef2-043f66981419"). InnerVolumeSpecName "kube-api-access-btplg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:45:56.696578 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.696533 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cae76037-9035-4b57-aaae-fa818a2dc7c2-plugins-config-volume\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:45:56.696578 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.696571 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cf0765e-2f30-4cb3-83e3-9d322281bf04-maas-api-tls\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:45:56.696578 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.696582 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ad201f7-0caa-4e24-aef2-043f66981419-plugins-config-volume\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:45:56.696819 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.696592 2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf0765e-2f30-4cb3-83e3-9d322281bf04-openshift-service-ca\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:45:56.696819 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.696604 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btplg\" (UniqueName: \"kubernetes.io/projected/9ad201f7-0caa-4e24-aef2-043f66981419-kube-api-access-btplg\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:45:56.696819 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.696617 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdl8v\" (UniqueName: \"kubernetes.io/projected/0cf0765e-2f30-4cb3-83e3-9d322281bf04-kube-api-access-sdl8v\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:45:56.696819 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.696626 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f87gb\" (UniqueName: \"kubernetes.io/projected/cae76037-9035-4b57-aaae-fa818a2dc7c2-kube-api-access-f87gb\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:45:56.757038 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.756944 2564 generic.go:356] "Generic (PLEG): container finished" podID="0cf0765e-2f30-4cb3-83e3-9d322281bf04" containerID="084daf29193e14664e5294df25733147bbb194b11a6c5bbacc149712e89bf793" exitCode=0 Jul 30 01:45:56.757038 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.756995 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" Jul 30 01:45:56.757038 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.757019 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" event={"ID":"0cf0765e-2f30-4cb3-83e3-9d322281bf04","Type":"ContainerDied","Data":"084daf29193e14664e5294df25733147bbb194b11a6c5bbacc149712e89bf793"} Jul 30 01:45:56.757299 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.757078 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8" event={"ID":"0cf0765e-2f30-4cb3-83e3-9d322281bf04","Type":"ContainerDied","Data":"62dd218733c0e06f135bb2b6992ec9f2ecb5427c4e59e9168ca719cf261ea090"} Jul 30 01:45:56.757299 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.757101 2564 scope.go:117] "RemoveContainer" containerID="084daf29193e14664e5294df25733147bbb194b11a6c5bbacc149712e89bf793" Jul 30 01:45:56.758432 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.758411 2564 generic.go:356] "Generic (PLEG): container finished" podID="9ad201f7-0caa-4e24-aef2-043f66981419" containerID="4c290616d82f1564f6b772c2f317d1e3c48417db01ecd1256a20cc3268a2663d" exitCode=0 Jul 30 01:45:56.758563 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.758456 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" Jul 30 01:45:56.758563 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.758478 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" event={"ID":"9ad201f7-0caa-4e24-aef2-043f66981419","Type":"ContainerDied","Data":"4c290616d82f1564f6b772c2f317d1e3c48417db01ecd1256a20cc3268a2663d"} Jul 30 01:45:56.758563 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.758524 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8" event={"ID":"9ad201f7-0caa-4e24-aef2-043f66981419","Type":"ContainerDied","Data":"206a013fc12956cb50d44517250e5f5f4ef8bc991087b3ac3f7f3fb26237a656"} Jul 30 01:45:56.760026 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.759940 2564 generic.go:356] "Generic (PLEG): container finished" podID="cae76037-9035-4b57-aaae-fa818a2dc7c2" containerID="7ab8d7b0873c531776f22ffca5ad9120b68ebcaeec3aed290f364a6cf5559abe" exitCode=0 Jul 30 01:45:56.760026 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.759969 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" event={"ID":"cae76037-9035-4b57-aaae-fa818a2dc7c2","Type":"ContainerDied","Data":"7ab8d7b0873c531776f22ffca5ad9120b68ebcaeec3aed290f364a6cf5559abe"} Jul 30 01:45:56.760307 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.759993 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" Jul 30 01:45:56.761427 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.760700 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv" event={"ID":"cae76037-9035-4b57-aaae-fa818a2dc7c2","Type":"ContainerDied","Data":"184e7858e40770610730710f7e5decd91440e1aed07938b92e41c1fb06830dcf"} Jul 30 01:45:56.775691 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.775661 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:45:56.776261 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.776243 2564 scope.go:117] "RemoveContainer" containerID="084daf29193e14664e5294df25733147bbb194b11a6c5bbacc149712e89bf793" Jul 30 01:45:56.776579 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:45:56.776555 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"084daf29193e14664e5294df25733147bbb194b11a6c5bbacc149712e89bf793\": container with ID starting with 084daf29193e14664e5294df25733147bbb194b11a6c5bbacc149712e89bf793 not found: ID does not exist" containerID="084daf29193e14664e5294df25733147bbb194b11a6c5bbacc149712e89bf793" Jul 30 01:45:56.776669 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.776587 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"084daf29193e14664e5294df25733147bbb194b11a6c5bbacc149712e89bf793"} err="failed to get container status \"084daf29193e14664e5294df25733147bbb194b11a6c5bbacc149712e89bf793\": rpc error: code = NotFound desc = could not find container \"084daf29193e14664e5294df25733147bbb194b11a6c5bbacc149712e89bf793\": container with ID starting with 084daf29193e14664e5294df25733147bbb194b11a6c5bbacc149712e89bf793 not found: ID does not exist" Jul 30 01:45:56.776669 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.776609 2564 scope.go:117] "RemoveContainer" containerID="4c290616d82f1564f6b772c2f317d1e3c48417db01ecd1256a20cc3268a2663d" Jul 30 01:45:56.786425 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.786402 2564 scope.go:117] "RemoveContainer" containerID="4c290616d82f1564f6b772c2f317d1e3c48417db01ecd1256a20cc3268a2663d" Jul 30 01:45:56.786707 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:45:56.786684 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c290616d82f1564f6b772c2f317d1e3c48417db01ecd1256a20cc3268a2663d\": container with ID starting with 4c290616d82f1564f6b772c2f317d1e3c48417db01ecd1256a20cc3268a2663d not found: ID does not exist" containerID="4c290616d82f1564f6b772c2f317d1e3c48417db01ecd1256a20cc3268a2663d" Jul 30 01:45:56.786755 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.786716 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c290616d82f1564f6b772c2f317d1e3c48417db01ecd1256a20cc3268a2663d"} err="failed to get container status \"4c290616d82f1564f6b772c2f317d1e3c48417db01ecd1256a20cc3268a2663d\": rpc error: code = NotFound desc = could not find container \"4c290616d82f1564f6b772c2f317d1e3c48417db01ecd1256a20cc3268a2663d\": container with ID starting with 4c290616d82f1564f6b772c2f317d1e3c48417db01ecd1256a20cc3268a2663d not found: ID does not exist" Jul 30 01:45:56.786755 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.786734 2564 scope.go:117] "RemoveContainer" containerID="7ab8d7b0873c531776f22ffca5ad9120b68ebcaeec3aed290f364a6cf5559abe" Jul 30 01:45:56.788850 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.788829 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8"] Jul 30 01:45:56.796258 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.796236 2564 scope.go:117] "RemoveContainer" containerID="7ab8d7b0873c531776f22ffca5ad9120b68ebcaeec3aed290f364a6cf5559abe" Jul 30 01:45:56.796613 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:45:56.796591 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ab8d7b0873c531776f22ffca5ad9120b68ebcaeec3aed290f364a6cf5559abe\": container with ID starting with 7ab8d7b0873c531776f22ffca5ad9120b68ebcaeec3aed290f364a6cf5559abe not found: ID does not exist" containerID="7ab8d7b0873c531776f22ffca5ad9120b68ebcaeec3aed290f364a6cf5559abe" Jul 30 01:45:56.796679 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.796624 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ab8d7b0873c531776f22ffca5ad9120b68ebcaeec3aed290f364a6cf5559abe"} err="failed to get container status \"7ab8d7b0873c531776f22ffca5ad9120b68ebcaeec3aed290f364a6cf5559abe\": rpc error: code = NotFound desc = could not find container \"7ab8d7b0873c531776f22ffca5ad9120b68ebcaeec3aed290f364a6cf5559abe\": container with ID starting with 7ab8d7b0873c531776f22ffca5ad9120b68ebcaeec3aed290f364a6cf5559abe not found: ID does not exist" Jul 30 01:45:56.797313 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.797293 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-06c66b-76dd44899f-rrjr8"] Jul 30 01:45:56.807638 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.807605 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8"] Jul 30 01:45:56.813038 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.813013 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-06c66b-854885667d-v5nl8"] Jul 30 01:45:56.823537 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.823506 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv"] Jul 30 01:45:56.826113 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:56.826088 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-06c66b-bfc79585b-zpffv"] Jul 30 01:45:57.668656 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:57.668621 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cf0765e-2f30-4cb3-83e3-9d322281bf04" path="/var/lib/kubelet/pods/0cf0765e-2f30-4cb3-83e3-9d322281bf04/volumes" Jul 30 01:45:57.669044 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:57.669013 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ad201f7-0caa-4e24-aef2-043f66981419" path="/var/lib/kubelet/pods/9ad201f7-0caa-4e24-aef2-043f66981419/volumes" Jul 30 01:45:57.669348 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:45:57.669333 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cae76037-9035-4b57-aaae-fa818a2dc7c2" path="/var/lib/kubelet/pods/cae76037-9035-4b57-aaae-fa818a2dc7c2/volumes" Jul 30 01:46:02.494062 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:02.494031 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:46:02.594202 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:02.594166 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:46:02.754266 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:02.754176 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq"] Jul 30 01:46:02.754441 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:02.754410 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" podUID="bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" containerName="istio-proxy" containerID="cri-o://c177201cb95f158626712acaddd0285673903140c8e6b6da9ca29fba2efb3a30" gracePeriod=30 Jul 30 01:46:02.773854 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:02.773818 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:46:02.988905 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:02.988860 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:46:08.023570 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.023505 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:46:08.116852 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.116801 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-workload-certs\") pod \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " Jul 30 01:46:08.116852 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.116854 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-data\") pod \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " Jul 30 01:46:08.117106 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.116918 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istiod-ca-cert\") pod \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " Jul 30 01:46:08.117106 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.116962 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-workload-socket\") pod \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " Jul 30 01:46:08.117106 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.117043 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcsnj\" (UniqueName: \"kubernetes.io/projected/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-kube-api-access-dcsnj\") pod \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " Jul 30 01:46:08.117106 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.117060 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-token\") pod \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " Jul 30 01:46:08.117106 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.117089 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-podinfo\") pod \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " Jul 30 01:46:08.117361 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.117140 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-credential-socket\") pod \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " Jul 30 01:46:08.117509 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.117193 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" (UID: "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:46:08.117600 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.117410 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" (UID: "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:46:08.117600 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.117442 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" (UID: "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:46:08.117600 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.117444 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" (UID: "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:46:08.117600 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.117567 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-envoy\") pod \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\" (UID: \"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee\") " Jul 30 01:46:08.117957 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.117935 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-data" (OuterVolumeSpecName: "istio-data") pod "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" (UID: "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:46:08.118054 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.117997 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-workload-socket\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:46:08.118054 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.118018 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-credential-socket\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:46:08.118054 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.118034 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-workload-certs\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:46:08.118054 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.118047 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istiod-ca-cert\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:46:08.119407 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.119379 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-kube-api-access-dcsnj" (OuterVolumeSpecName: "kube-api-access-dcsnj") pod "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" (UID: "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee"). InnerVolumeSpecName "kube-api-access-dcsnj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:46:08.119525 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.119471 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" (UID: "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 01:46:08.119760 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.119746 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" (UID: "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:46:08.119808 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.119781 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-token" (OuterVolumeSpecName: "istio-token") pod "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" (UID: "bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:46:08.223720 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.223666 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-podinfo\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:46:08.223951 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.223733 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-envoy\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:46:08.223951 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.223750 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-data\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:46:08.223951 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.223764 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcsnj\" (UniqueName: \"kubernetes.io/projected/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-kube-api-access-dcsnj\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:46:08.223951 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.223778 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee-istio-token\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:46:08.833718 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.833688 2564 generic.go:356] "Generic (PLEG): container finished" podID="bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" containerID="c177201cb95f158626712acaddd0285673903140c8e6b6da9ca29fba2efb3a30" exitCode=0 Jul 30 01:46:08.833921 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.833752 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" event={"ID":"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee","Type":"ContainerDied","Data":"c177201cb95f158626712acaddd0285673903140c8e6b6da9ca29fba2efb3a30"} Jul 30 01:46:08.833921 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.833792 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" event={"ID":"bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee","Type":"ContainerDied","Data":"8feedf7b4e3ee1adf7658b07e3f9034e738ecf3bc909f2c289ae87291a950862"} Jul 30 01:46:08.833921 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.833816 2564 scope.go:117] "RemoveContainer" containerID="c177201cb95f158626712acaddd0285673903140c8e6b6da9ca29fba2efb3a30" Jul 30 01:46:08.833921 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.833754 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq" Jul 30 01:46:08.844282 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.844259 2564 scope.go:117] "RemoveContainer" containerID="08c871def0f81e0785ae0beeea7bded8e8cbcc0c1a5b688c3f4e8fb7667cdfba" Jul 30 01:46:08.853883 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.853854 2564 scope.go:117] "RemoveContainer" containerID="c177201cb95f158626712acaddd0285673903140c8e6b6da9ca29fba2efb3a30" Jul 30 01:46:08.854277 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:46:08.854248 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c177201cb95f158626712acaddd0285673903140c8e6b6da9ca29fba2efb3a30\": container with ID starting with c177201cb95f158626712acaddd0285673903140c8e6b6da9ca29fba2efb3a30 not found: ID does not exist" containerID="c177201cb95f158626712acaddd0285673903140c8e6b6da9ca29fba2efb3a30" Jul 30 01:46:08.854393 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.854287 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c177201cb95f158626712acaddd0285673903140c8e6b6da9ca29fba2efb3a30"} err="failed to get container status \"c177201cb95f158626712acaddd0285673903140c8e6b6da9ca29fba2efb3a30\": rpc error: code = NotFound desc = could not find container \"c177201cb95f158626712acaddd0285673903140c8e6b6da9ca29fba2efb3a30\": container with ID starting with c177201cb95f158626712acaddd0285673903140c8e6b6da9ca29fba2efb3a30 not found: ID does not exist" Jul 30 01:46:08.854393 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.854314 2564 scope.go:117] "RemoveContainer" containerID="08c871def0f81e0785ae0beeea7bded8e8cbcc0c1a5b688c3f4e8fb7667cdfba" Jul 30 01:46:08.854671 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:46:08.854653 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08c871def0f81e0785ae0beeea7bded8e8cbcc0c1a5b688c3f4e8fb7667cdfba\": container with ID starting with 08c871def0f81e0785ae0beeea7bded8e8cbcc0c1a5b688c3f4e8fb7667cdfba not found: ID does not exist" containerID="08c871def0f81e0785ae0beeea7bded8e8cbcc0c1a5b688c3f4e8fb7667cdfba" Jul 30 01:46:08.854714 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.854681 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08c871def0f81e0785ae0beeea7bded8e8cbcc0c1a5b688c3f4e8fb7667cdfba"} err="failed to get container status \"08c871def0f81e0785ae0beeea7bded8e8cbcc0c1a5b688c3f4e8fb7667cdfba\": rpc error: code = NotFound desc = could not find container \"08c871def0f81e0785ae0beeea7bded8e8cbcc0c1a5b688c3f4e8fb7667cdfba\": container with ID starting with 08c871def0f81e0785ae0beeea7bded8e8cbcc0c1a5b688c3f4e8fb7667cdfba not found: ID does not exist" Jul 30 01:46:08.856595 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.856558 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq"] Jul 30 01:46:08.859825 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:08.859802 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-06c66b-openshift-default-c96675c74-pffmq"] Jul 30 01:46:09.669547 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:09.669510 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" path="/var/lib/kubelet/pods/bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee/volumes" Jul 30 01:46:22.076779 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:22.076728 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-09b0d2-29756265-hjgqg"] Jul 30 01:46:22.079171 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:22.079143 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-09b0d2-29756265-hjgqg"] Jul 30 01:46:22.082135 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:22.082107 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:46:23.243948 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:23.243917 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:46:23.675668 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:23.671925 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da0d2adb-62ef-4506-b382-39eee9a084be" path="/var/lib/kubelet/pods/da0d2adb-62ef-4506-b382-39eee9a084be/volumes" Jul 30 01:46:27.646563 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:46:27.646528 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:04.336877 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:04.336846 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:04.702673 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:04.702580 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:10.120432 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:10.120393 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:24.221497 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.221459 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:24.281514 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.281475 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:24.497344 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.497247 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:24.720144 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.720110 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:24.844088 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.844047 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw"] Jul 30 01:47:24.844480 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.844466 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" containerName="istio-proxy" Jul 30 01:47:24.844528 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.844482 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" containerName="istio-proxy" Jul 30 01:47:24.844528 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.844495 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ad201f7-0caa-4e24-aef2-043f66981419" containerName="payload-processing" Jul 30 01:47:24.844528 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.844501 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ad201f7-0caa-4e24-aef2-043f66981419" containerName="payload-processing" Jul 30 01:47:24.844528 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.844515 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cae76037-9035-4b57-aaae-fa818a2dc7c2" containerName="payload-pre-processing" Jul 30 01:47:24.844528 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.844522 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cae76037-9035-4b57-aaae-fa818a2dc7c2" containerName="payload-pre-processing" Jul 30 01:47:24.844680 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.844530 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cf0765e-2f30-4cb3-83e3-9d322281bf04" containerName="maas-api" Jul 30 01:47:24.844680 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.844536 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cf0765e-2f30-4cb3-83e3-9d322281bf04" containerName="maas-api" Jul 30 01:47:24.844680 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.844588 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" containerName="istio-proxy" Jul 30 01:47:24.844680 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.844596 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ad201f7-0caa-4e24-aef2-043f66981419" containerName="payload-processing" Jul 30 01:47:24.844680 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.844601 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" containerName="istio-proxy" Jul 30 01:47:24.844680 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.844608 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cae76037-9035-4b57-aaae-fa818a2dc7c2" containerName="payload-pre-processing" Jul 30 01:47:24.844680 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.844615 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cf0765e-2f30-4cb3-83e3-9d322281bf04" containerName="maas-api" Jul 30 01:47:24.847794 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.847778 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw" Jul 30 01:47:24.850110 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.850091 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\"" Jul 30 01:47:24.856460 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.856427 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw"] Jul 30 01:47:24.978952 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:24.978891 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3569e2b2-c35d-44e3-a7e0-63438733aba6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw\" (UID: \"3569e2b2-c35d-44e3-a7e0-63438733aba6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw" Jul 30 01:47:25.079560 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:25.079505 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3569e2b2-c35d-44e3-a7e0-63438733aba6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw\" (UID: \"3569e2b2-c35d-44e3-a7e0-63438733aba6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw" Jul 30 01:47:25.080159 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:25.080139 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3569e2b2-c35d-44e3-a7e0-63438733aba6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw\" (UID: \"3569e2b2-c35d-44e3-a7e0-63438733aba6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw" Jul 30 01:47:25.159757 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:25.159670 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw" Jul 30 01:47:25.298973 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:25.298946 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw"] Jul 30 01:47:25.300940 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:47:25.300906 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3569e2b2_c35d_44e3_a7e0_63438733aba6.slice/crio-27940422be85daf9a5465362b551a316686ca6516341a64500ea984687c71b61 WatchSource:0}: Error finding container 27940422be85daf9a5465362b551a316686ca6516341a64500ea984687c71b61: Status 404 returned error can't find the container with id 27940422be85daf9a5465362b551a316686ca6516341a64500ea984687c71b61 Jul 30 01:47:26.281651 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:26.281617 2564 generic.go:356] "Generic (PLEG): container finished" podID="3569e2b2-c35d-44e3-a7e0-63438733aba6" containerID="8bc099984db59fb7bc4775ea02a099e0e15f150b2a532e5311fb821359368f69" exitCode=0 Jul 30 01:47:26.281839 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:26.281705 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw" event={"ID":"3569e2b2-c35d-44e3-a7e0-63438733aba6","Type":"ContainerDied","Data":"8bc099984db59fb7bc4775ea02a099e0e15f150b2a532e5311fb821359368f69"} Jul 30 01:47:26.281839 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:26.281760 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw" event={"ID":"3569e2b2-c35d-44e3-a7e0-63438733aba6","Type":"ContainerStarted","Data":"27940422be85daf9a5465362b551a316686ca6516341a64500ea984687c71b61"} Jul 30 01:47:27.431033 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:27.431007 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw" Jul 30 01:47:27.502333 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:27.502285 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3569e2b2-c35d-44e3-a7e0-63438733aba6-maas-api-service-ca\") pod \"3569e2b2-c35d-44e3-a7e0-63438733aba6\" (UID: \"3569e2b2-c35d-44e3-a7e0-63438733aba6\") " Jul 30 01:47:27.502682 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:27.502658 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3569e2b2-c35d-44e3-a7e0-63438733aba6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3569e2b2-c35d-44e3-a7e0-63438733aba6" (UID: "3569e2b2-c35d-44e3-a7e0-63438733aba6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:47:27.603492 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:27.603437 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3569e2b2-c35d-44e3-a7e0-63438733aba6-maas-api-service-ca\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:47:28.294747 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:28.294715 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw" Jul 30 01:47:28.294946 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:28.294757 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw" event={"ID":"3569e2b2-c35d-44e3-a7e0-63438733aba6","Type":"ContainerDied","Data":"27940422be85daf9a5465362b551a316686ca6516341a64500ea984687c71b61"} Jul 30 01:47:28.294946 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:28.294808 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27940422be85daf9a5465362b551a316686ca6516341a64500ea984687c71b61" Jul 30 01:47:35.235200 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.235159 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz"] Jul 30 01:47:35.235811 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.235452 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" podUID="92472797-0d13-4e6f-b636-9949b5e9f238" containerName="maas-api" containerID="cri-o://a5eda8d8951bd0a20564f9e3f6370a55bc1d78ae46a5c55aa1c103ee82ef7ed2" gracePeriod=30 Jul 30 01:47:35.267130 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.267092 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-b64c39-29756265-9cks8"] Jul 30 01:47:35.270398 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.270369 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-b64c39-29756265-9cks8"] Jul 30 01:47:35.285734 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.285690 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:35.303847 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.303805 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td"] Jul 30 01:47:35.304187 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.304071 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" podUID="558c0c9a-d249-4941-9ec6-ef7df138048c" containerName="payload-pre-processing" containerID="cri-o://f225143ee20e039c7010faf1ee0f6c76951d99233ac33f016d7909e8abeb9237" gracePeriod=30 Jul 30 01:47:35.356166 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.356129 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:35.441058 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.441020 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:35.441439 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.441416 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw"] Jul 30 01:47:35.445741 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.445709 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b64c39-dkqvw"] Jul 30 01:47:35.570004 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.569974 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:47:35.601983 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.601957 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" Jul 30 01:47:35.669004 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.668972 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3569e2b2-c35d-44e3-a7e0-63438733aba6" path="/var/lib/kubelet/pods/3569e2b2-c35d-44e3-a7e0-63438733aba6/volumes" Jul 30 01:47:35.669297 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.669284 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cad0cf5c-b02f-4164-b732-888374852831" path="/var/lib/kubelet/pods/cad0cf5c-b02f-4164-b732-888374852831/volumes" Jul 30 01:47:35.694683 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.694650 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/558c0c9a-d249-4941-9ec6-ef7df138048c-plugins-config-volume\") pod \"558c0c9a-d249-4941-9ec6-ef7df138048c\" (UID: \"558c0c9a-d249-4941-9ec6-ef7df138048c\") " Jul 30 01:47:35.694863 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.694698 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92472797-0d13-4e6f-b636-9949b5e9f238-maas-api-tls\") pod \"92472797-0d13-4e6f-b636-9949b5e9f238\" (UID: \"92472797-0d13-4e6f-b636-9949b5e9f238\") " Jul 30 01:47:35.694863 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.694770 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-457hl\" (UniqueName: \"kubernetes.io/projected/92472797-0d13-4e6f-b636-9949b5e9f238-kube-api-access-457hl\") pod \"92472797-0d13-4e6f-b636-9949b5e9f238\" (UID: \"92472797-0d13-4e6f-b636-9949b5e9f238\") " Jul 30 01:47:35.694863 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.694795 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbqbq\" (UniqueName: \"kubernetes.io/projected/558c0c9a-d249-4941-9ec6-ef7df138048c-kube-api-access-wbqbq\") pod \"558c0c9a-d249-4941-9ec6-ef7df138048c\" (UID: \"558c0c9a-d249-4941-9ec6-ef7df138048c\") " Jul 30 01:47:35.695038 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.694865 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92472797-0d13-4e6f-b636-9949b5e9f238-openshift-service-ca\") pod \"92472797-0d13-4e6f-b636-9949b5e9f238\" (UID: \"92472797-0d13-4e6f-b636-9949b5e9f238\") " Jul 30 01:47:35.695161 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.695126 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/558c0c9a-d249-4941-9ec6-ef7df138048c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "558c0c9a-d249-4941-9ec6-ef7df138048c" (UID: "558c0c9a-d249-4941-9ec6-ef7df138048c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:47:35.695371 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.695347 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92472797-0d13-4e6f-b636-9949b5e9f238-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "92472797-0d13-4e6f-b636-9949b5e9f238" (UID: "92472797-0d13-4e6f-b636-9949b5e9f238"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:47:35.695430 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.695377 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/558c0c9a-d249-4941-9ec6-ef7df138048c-plugins-config-volume\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:47:35.696941 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.696918 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92472797-0d13-4e6f-b636-9949b5e9f238-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "92472797-0d13-4e6f-b636-9949b5e9f238" (UID: "92472797-0d13-4e6f-b636-9949b5e9f238"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 01:47:35.697015 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.696974 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/558c0c9a-d249-4941-9ec6-ef7df138048c-kube-api-access-wbqbq" (OuterVolumeSpecName: "kube-api-access-wbqbq") pod "558c0c9a-d249-4941-9ec6-ef7df138048c" (UID: "558c0c9a-d249-4941-9ec6-ef7df138048c"). InnerVolumeSpecName "kube-api-access-wbqbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:47:35.697015 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.697009 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92472797-0d13-4e6f-b636-9949b5e9f238-kube-api-access-457hl" (OuterVolumeSpecName: "kube-api-access-457hl") pod "92472797-0d13-4e6f-b636-9949b5e9f238" (UID: "92472797-0d13-4e6f-b636-9949b5e9f238"). InnerVolumeSpecName "kube-api-access-457hl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:47:35.796774 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.796678 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-457hl\" (UniqueName: \"kubernetes.io/projected/92472797-0d13-4e6f-b636-9949b5e9f238-kube-api-access-457hl\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:47:35.796774 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.796707 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbqbq\" (UniqueName: \"kubernetes.io/projected/558c0c9a-d249-4941-9ec6-ef7df138048c-kube-api-access-wbqbq\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:47:35.796774 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.796717 2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92472797-0d13-4e6f-b636-9949b5e9f238-openshift-service-ca\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:47:35.796774 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.796727 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92472797-0d13-4e6f-b636-9949b5e9f238-maas-api-tls\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:47:35.824954 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:35.824857 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:36.341826 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.341791 2564 generic.go:356] "Generic (PLEG): container finished" podID="558c0c9a-d249-4941-9ec6-ef7df138048c" containerID="f225143ee20e039c7010faf1ee0f6c76951d99233ac33f016d7909e8abeb9237" exitCode=0 Jul 30 01:47:36.342280 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.341837 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" Jul 30 01:47:36.342280 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.341876 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" event={"ID":"558c0c9a-d249-4941-9ec6-ef7df138048c","Type":"ContainerDied","Data":"f225143ee20e039c7010faf1ee0f6c76951d99233ac33f016d7909e8abeb9237"} Jul 30 01:47:36.342280 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.341934 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td" event={"ID":"558c0c9a-d249-4941-9ec6-ef7df138048c","Type":"ContainerDied","Data":"bbcd099d7bd2ac952e4befe59ea29212ece5304cce4b2aa9948ff756c1094e4f"} Jul 30 01:47:36.342280 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.341963 2564 scope.go:117] "RemoveContainer" containerID="f225143ee20e039c7010faf1ee0f6c76951d99233ac33f016d7909e8abeb9237" Jul 30 01:47:36.343346 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.343240 2564 generic.go:356] "Generic (PLEG): container finished" podID="92472797-0d13-4e6f-b636-9949b5e9f238" containerID="a5eda8d8951bd0a20564f9e3f6370a55bc1d78ae46a5c55aa1c103ee82ef7ed2" exitCode=0 Jul 30 01:47:36.343346 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.343285 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" Jul 30 01:47:36.343346 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.343330 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" event={"ID":"92472797-0d13-4e6f-b636-9949b5e9f238","Type":"ContainerDied","Data":"a5eda8d8951bd0a20564f9e3f6370a55bc1d78ae46a5c55aa1c103ee82ef7ed2"} Jul 30 01:47:36.343474 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.343378 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz" event={"ID":"92472797-0d13-4e6f-b636-9949b5e9f238","Type":"ContainerDied","Data":"5df8b7b04db3c3eb1e96c6a351fd01b40f7ce1567a9a1c81dfb9f3e3f95babac"} Jul 30 01:47:36.352823 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.352799 2564 scope.go:117] "RemoveContainer" containerID="f225143ee20e039c7010faf1ee0f6c76951d99233ac33f016d7909e8abeb9237" Jul 30 01:47:36.353177 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:47:36.353153 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f225143ee20e039c7010faf1ee0f6c76951d99233ac33f016d7909e8abeb9237\": container with ID starting with f225143ee20e039c7010faf1ee0f6c76951d99233ac33f016d7909e8abeb9237 not found: ID does not exist" containerID="f225143ee20e039c7010faf1ee0f6c76951d99233ac33f016d7909e8abeb9237" Jul 30 01:47:36.353275 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.353190 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f225143ee20e039c7010faf1ee0f6c76951d99233ac33f016d7909e8abeb9237"} err="failed to get container status \"f225143ee20e039c7010faf1ee0f6c76951d99233ac33f016d7909e8abeb9237\": rpc error: code = NotFound desc = could not find container \"f225143ee20e039c7010faf1ee0f6c76951d99233ac33f016d7909e8abeb9237\": container with ID starting with f225143ee20e039c7010faf1ee0f6c76951d99233ac33f016d7909e8abeb9237 not found: ID does not exist" Jul 30 01:47:36.353275 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.353213 2564 scope.go:117] "RemoveContainer" containerID="a5eda8d8951bd0a20564f9e3f6370a55bc1d78ae46a5c55aa1c103ee82ef7ed2" Jul 30 01:47:36.361817 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.361794 2564 scope.go:117] "RemoveContainer" containerID="a5eda8d8951bd0a20564f9e3f6370a55bc1d78ae46a5c55aa1c103ee82ef7ed2" Jul 30 01:47:36.362110 ip-10-0-136-247 kubenswrapper[2564]: E0730 01:47:36.362092 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5eda8d8951bd0a20564f9e3f6370a55bc1d78ae46a5c55aa1c103ee82ef7ed2\": container with ID starting with a5eda8d8951bd0a20564f9e3f6370a55bc1d78ae46a5c55aa1c103ee82ef7ed2 not found: ID does not exist" containerID="a5eda8d8951bd0a20564f9e3f6370a55bc1d78ae46a5c55aa1c103ee82ef7ed2" Jul 30 01:47:36.362149 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.362118 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5eda8d8951bd0a20564f9e3f6370a55bc1d78ae46a5c55aa1c103ee82ef7ed2"} err="failed to get container status \"a5eda8d8951bd0a20564f9e3f6370a55bc1d78ae46a5c55aa1c103ee82ef7ed2\": rpc error: code = NotFound desc = could not find container \"a5eda8d8951bd0a20564f9e3f6370a55bc1d78ae46a5c55aa1c103ee82ef7ed2\": container with ID starting with a5eda8d8951bd0a20564f9e3f6370a55bc1d78ae46a5c55aa1c103ee82ef7ed2 not found: ID does not exist" Jul 30 01:47:36.366787 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.366759 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz"] Jul 30 01:47:36.370379 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.370353 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b64c39-77dbb6cb59-ldcnz"] Jul 30 01:47:36.379664 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.379636 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td"] Jul 30 01:47:36.387044 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:36.387019 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b64c39-c9b98c4fb-kk9td"] Jul 30 01:47:37.668988 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:37.668953 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="558c0c9a-d249-4941-9ec6-ef7df138048c" path="/var/lib/kubelet/pods/558c0c9a-d249-4941-9ec6-ef7df138048c/volumes" Jul 30 01:47:37.669383 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:37.669363 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92472797-0d13-4e6f-b636-9949b5e9f238" path="/var/lib/kubelet/pods/92472797-0d13-4e6f-b636-9949b5e9f238/volumes" Jul 30 01:47:41.426347 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:41.426263 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:41.938744 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:41.938710 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:41.988060 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:41.988035 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:42.060597 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.060555 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr"] Jul 30 01:47:42.061131 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.061103 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" containerName="istio-proxy" Jul 30 01:47:42.061190 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.061136 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb0b8046-a1c9-4494-b4d9-8b87bb2ef6ee" containerName="istio-proxy" Jul 30 01:47:42.061190 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.061164 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3569e2b2-c35d-44e3-a7e0-63438733aba6" containerName="revoke-keys" Jul 30 01:47:42.061190 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.061170 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3569e2b2-c35d-44e3-a7e0-63438733aba6" containerName="revoke-keys" Jul 30 01:47:42.061190 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.061180 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92472797-0d13-4e6f-b636-9949b5e9f238" containerName="maas-api" Jul 30 01:47:42.061190 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.061186 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="92472797-0d13-4e6f-b636-9949b5e9f238" containerName="maas-api" Jul 30 01:47:42.061351 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.061194 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="558c0c9a-d249-4941-9ec6-ef7df138048c" containerName="payload-pre-processing" Jul 30 01:47:42.061351 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.061199 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="558c0c9a-d249-4941-9ec6-ef7df138048c" containerName="payload-pre-processing" Jul 30 01:47:42.061351 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.061262 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="92472797-0d13-4e6f-b636-9949b5e9f238" containerName="maas-api" Jul 30 01:47:42.061351 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.061269 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="558c0c9a-d249-4941-9ec6-ef7df138048c" containerName="payload-pre-processing" Jul 30 01:47:42.061351 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.061279 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3569e2b2-c35d-44e3-a7e0-63438733aba6" containerName="revoke-keys" Jul 30 01:47:42.065816 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.065797 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr" Jul 30 01:47:42.068122 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.068098 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\"" Jul 30 01:47:42.069103 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.069074 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 30 01:47:42.072169 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.072142 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr"] Jul 30 01:47:42.263251 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.263152 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/663cfac9-7b75-49cd-8244-884ca6ea32bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr\" (UID: \"663cfac9-7b75-49cd-8244-884ca6ea32bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr" Jul 30 01:47:42.364770 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.364727 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/663cfac9-7b75-49cd-8244-884ca6ea32bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr\" (UID: \"663cfac9-7b75-49cd-8244-884ca6ea32bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr" Jul 30 01:47:42.365411 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.365390 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/663cfac9-7b75-49cd-8244-884ca6ea32bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr\" (UID: \"663cfac9-7b75-49cd-8244-884ca6ea32bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr" Jul 30 01:47:42.378613 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.378584 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr" Jul 30 01:47:42.720137 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:42.720107 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr"] Jul 30 01:47:42.721700 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:47:42.721671 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod663cfac9_7b75_49cd_8244_884ca6ea32bd.slice/crio-b3816a3403a3f2b86dbd594afa9a1d5290a0f0409c9d3dc8de1a232d67779647 WatchSource:0}: Error finding container b3816a3403a3f2b86dbd594afa9a1d5290a0f0409c9d3dc8de1a232d67779647: Status 404 returned error can't find the container with id b3816a3403a3f2b86dbd594afa9a1d5290a0f0409c9d3dc8de1a232d67779647 Jul 30 01:47:43.385683 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:43.385648 2564 generic.go:356] "Generic (PLEG): container finished" podID="663cfac9-7b75-49cd-8244-884ca6ea32bd" containerID="7a565e85989f143210105328bcf2521cf98e9d2af92d37e1cd310fe0ae7a447c" exitCode=0 Jul 30 01:47:43.385866 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:43.385728 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr" event={"ID":"663cfac9-7b75-49cd-8244-884ca6ea32bd","Type":"ContainerDied","Data":"7a565e85989f143210105328bcf2521cf98e9d2af92d37e1cd310fe0ae7a447c"} Jul 30 01:47:43.385866 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:43.385780 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr" event={"ID":"663cfac9-7b75-49cd-8244-884ca6ea32bd","Type":"ContainerStarted","Data":"b3816a3403a3f2b86dbd594afa9a1d5290a0f0409c9d3dc8de1a232d67779647"} Jul 30 01:47:44.527878 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:44.527853 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr" Jul 30 01:47:44.581663 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:44.581621 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/663cfac9-7b75-49cd-8244-884ca6ea32bd-maas-api-service-ca\") pod \"663cfac9-7b75-49cd-8244-884ca6ea32bd\" (UID: \"663cfac9-7b75-49cd-8244-884ca6ea32bd\") " Jul 30 01:47:44.582013 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:44.581991 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/663cfac9-7b75-49cd-8244-884ca6ea32bd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "663cfac9-7b75-49cd-8244-884ca6ea32bd" (UID: "663cfac9-7b75-49cd-8244-884ca6ea32bd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:47:44.682405 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:44.682310 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/663cfac9-7b75-49cd-8244-884ca6ea32bd-maas-api-service-ca\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:47:45.398874 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:45.398844 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr" Jul 30 01:47:45.399103 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:45.398872 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr" event={"ID":"663cfac9-7b75-49cd-8244-884ca6ea32bd","Type":"ContainerDied","Data":"b3816a3403a3f2b86dbd594afa9a1d5290a0f0409c9d3dc8de1a232d67779647"} Jul 30 01:47:45.399103 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:45.398940 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3816a3403a3f2b86dbd594afa9a1d5290a0f0409c9d3dc8de1a232d67779647" Jul 30 01:47:52.476024 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:52.475991 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-55b321-29756265-tmcjz"] Jul 30 01:47:52.480175 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:52.480147 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-55b321-29756265-tmcjz"] Jul 30 01:47:52.481490 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:52.481455 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:52.581710 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:52.581677 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:52.642049 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:52.642013 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr"] Jul 30 01:47:52.644479 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:52.644452 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-55b321-nd5dr"] Jul 30 01:47:52.650254 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:52.650223 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:53.012192 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:53.012156 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:53.669506 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:53.669469 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="663cfac9-7b75-49cd-8244-884ca6ea32bd" path="/var/lib/kubelet/pods/663cfac9-7b75-49cd-8244-884ca6ea32bd/volumes" Jul 30 01:47:53.669889 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:53.669756 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b1f36c3-1e52-4b03-bc75-7643ae87d34c" path="/var/lib/kubelet/pods/9b1f36c3-1e52-4b03-bc75-7643ae87d34c/volumes" Jul 30 01:47:58.985961 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:58.985861 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:59.046622 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:59.046588 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:59.253557 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:59.253471 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:47:59.454624 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:47:59.454587 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:48:09.980352 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:09.980314 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fba93c-29756265-dh7xv"] Jul 30 01:48:09.984683 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:09.984642 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fba93c-29756265-dh7xv"] Jul 30 01:48:09.984858 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:09.984694 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:48:10.046178 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:10.046145 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:48:10.107238 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:10.107208 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:48:10.207346 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:10.207316 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:48:10.512944 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:10.512892 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:48:11.668798 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:11.668763 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7da35555-8dc2-47be-af2d-2f61f5df3b1c" path="/var/lib/kubelet/pods/7da35555-8dc2-47be-af2d-2f61f5df3b1c/volumes" Jul 30 01:48:16.481638 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:16.481600 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg"] Jul 30 01:48:16.482294 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:16.481813 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" podUID="c18af419-ea76-4774-b5b2-608b96926159" containerName="istio-proxy" containerID="cri-o://d8c18cc9a64b2274ae4da3c15898ed017b8a1e7510e55027058a1cf91da8a864" gracePeriod=30 Jul 30 01:48:16.619179 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:16.619144 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:48:16.977957 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:16.977889 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:48:17.039038 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:17.039002 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:48:20.294370 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:20.294325 2564 patch_prober.go:28] interesting pod/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 01:48:20.294778 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:20.294401 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" podUID="c18af419-ea76-4774-b5b2-608b96926159" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 01:48:21.619317 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.619283 2564 generic.go:356] "Generic (PLEG): container finished" podID="c18af419-ea76-4774-b5b2-608b96926159" containerID="d8c18cc9a64b2274ae4da3c15898ed017b8a1e7510e55027058a1cf91da8a864" exitCode=0 Jul 30 01:48:21.619724 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.619352 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" event={"ID":"c18af419-ea76-4774-b5b2-608b96926159","Type":"ContainerDied","Data":"d8c18cc9a64b2274ae4da3c15898ed017b8a1e7510e55027058a1cf91da8a864"} Jul 30 01:48:21.619724 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.619432 2564 scope.go:117] "RemoveContainer" containerID="0b2e5e9a38c7d9567428417335020cff6355ad8e1d7ac7ab50ebc93446e0aad7" Jul 30 01:48:21.754972 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.754852 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:48:21.948864 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.948821 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-workload-certs\") pod \"c18af419-ea76-4774-b5b2-608b96926159\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " Jul 30 01:48:21.949212 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.948871 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-workload-socket\") pod \"c18af419-ea76-4774-b5b2-608b96926159\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " Jul 30 01:48:21.949212 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.948937 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-istio-envoy\") pod \"c18af419-ea76-4774-b5b2-608b96926159\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " Jul 30 01:48:21.949212 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.948968 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c18af419-ea76-4774-b5b2-608b96926159-istio-token\") pod \"c18af419-ea76-4774-b5b2-608b96926159\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " Jul 30 01:48:21.949212 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.949015 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c18af419-ea76-4774-b5b2-608b96926159-istio-podinfo\") pod \"c18af419-ea76-4774-b5b2-608b96926159\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " Jul 30 01:48:21.949212 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.949039 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c18af419-ea76-4774-b5b2-608b96926159-istiod-ca-cert\") pod \"c18af419-ea76-4774-b5b2-608b96926159\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " Jul 30 01:48:21.949212 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.949099 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-istio-data\") pod \"c18af419-ea76-4774-b5b2-608b96926159\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " Jul 30 01:48:21.949212 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.949145 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzrwf\" (UniqueName: \"kubernetes.io/projected/c18af419-ea76-4774-b5b2-608b96926159-kube-api-access-lzrwf\") pod \"c18af419-ea76-4774-b5b2-608b96926159\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " Jul 30 01:48:21.949212 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.949166 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c18af419-ea76-4774-b5b2-608b96926159" (UID: "c18af419-ea76-4774-b5b2-608b96926159"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:48:21.949212 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.949184 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-credential-socket\") pod \"c18af419-ea76-4774-b5b2-608b96926159\" (UID: \"c18af419-ea76-4774-b5b2-608b96926159\") " Jul 30 01:48:21.949685 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.949226 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c18af419-ea76-4774-b5b2-608b96926159" (UID: "c18af419-ea76-4774-b5b2-608b96926159"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:48:21.949685 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.949452 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c18af419-ea76-4774-b5b2-608b96926159-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c18af419-ea76-4774-b5b2-608b96926159" (UID: "c18af419-ea76-4774-b5b2-608b96926159"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:48:21.949685 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.949484 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-workload-certs\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:48:21.949685 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.949502 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-workload-socket\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:48:21.949685 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.949659 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c18af419-ea76-4774-b5b2-608b96926159" (UID: "c18af419-ea76-4774-b5b2-608b96926159"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:48:21.950017 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.949989 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-istio-data" (OuterVolumeSpecName: "istio-data") pod "c18af419-ea76-4774-b5b2-608b96926159" (UID: "c18af419-ea76-4774-b5b2-608b96926159"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:48:21.951404 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.951378 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c18af419-ea76-4774-b5b2-608b96926159" (UID: "c18af419-ea76-4774-b5b2-608b96926159"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:48:21.951529 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.951405 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c18af419-ea76-4774-b5b2-608b96926159-kube-api-access-lzrwf" (OuterVolumeSpecName: "kube-api-access-lzrwf") pod "c18af419-ea76-4774-b5b2-608b96926159" (UID: "c18af419-ea76-4774-b5b2-608b96926159"). InnerVolumeSpecName "kube-api-access-lzrwf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:48:21.951529 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.951436 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c18af419-ea76-4774-b5b2-608b96926159-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c18af419-ea76-4774-b5b2-608b96926159" (UID: "c18af419-ea76-4774-b5b2-608b96926159"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 01:48:21.951529 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:21.951486 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c18af419-ea76-4774-b5b2-608b96926159-istio-token" (OuterVolumeSpecName: "istio-token") pod "c18af419-ea76-4774-b5b2-608b96926159" (UID: "c18af419-ea76-4774-b5b2-608b96926159"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:48:22.050804 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:22.050706 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-istio-envoy\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:48:22.050804 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:22.050742 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c18af419-ea76-4774-b5b2-608b96926159-istio-token\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:48:22.050804 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:22.050752 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c18af419-ea76-4774-b5b2-608b96926159-istio-podinfo\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:48:22.050804 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:22.050760 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c18af419-ea76-4774-b5b2-608b96926159-istiod-ca-cert\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:48:22.050804 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:22.050768 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-istio-data\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:48:22.050804 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:22.050777 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzrwf\" (UniqueName: \"kubernetes.io/projected/c18af419-ea76-4774-b5b2-608b96926159-kube-api-access-lzrwf\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:48:22.050804 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:22.050785 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18af419-ea76-4774-b5b2-608b96926159-credential-socket\") on node \"ip-10-0-136-247.ec2.internal\" DevicePath \"\"" Jul 30 01:48:22.625825 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:22.625790 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" event={"ID":"c18af419-ea76-4774-b5b2-608b96926159","Type":"ContainerDied","Data":"f2b368b75fd9711442442a1a2f20e4fa699b4b408b2c5dede6ba786162ae889b"} Jul 30 01:48:22.625825 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:22.625808 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg" Jul 30 01:48:22.626366 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:22.625860 2564 scope.go:117] "RemoveContainer" containerID="d8c18cc9a64b2274ae4da3c15898ed017b8a1e7510e55027058a1cf91da8a864" Jul 30 01:48:22.647366 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:22.647323 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg"] Jul 30 01:48:22.650990 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:22.650960 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-fba93c-openshift-default-5bb4f7dbb4-97mhg"] Jul 30 01:48:23.372759 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:23.372731 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b4b6684-htpnw_b2dfe990-5238-4ad7-8c27-43dfde9eda24/copy-manifests/0.log" Jul 30 01:48:23.378705 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:23.378672 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b4b6684-htpnw_b2dfe990-5238-4ad7-8c27-43dfde9eda24/manager/0.log" Jul 30 01:48:23.676243 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:23.676146 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c18af419-ea76-4774-b5b2-608b96926159" path="/var/lib/kubelet/pods/c18af419-ea76-4774-b5b2-608b96926159/volumes" Jul 30 01:48:24.043783 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:24.043701 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29756265-kwh67_eac78ae6-5093-4547-8010-d2c981284648/cleanup/0.log" Jul 30 01:48:24.921097 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:24.921070 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks_2c4ece51-398a-41c3-9cd8-87d4cce68a8b/util/0.log" Jul 30 01:48:24.926885 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:24.926860 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks_2c4ece51-398a-41c3-9cd8-87d4cce68a8b/pull/0.log" Jul 30 01:48:24.932580 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:24.932561 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks_2c4ece51-398a-41c3-9cd8-87d4cce68a8b/extract/0.log" Jul 30 01:48:25.041699 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:25.041671 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb_1503a956-5e65-4c0b-8d68-d932e789d4a6/util/0.log" Jul 30 01:48:25.047396 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:25.047372 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb_1503a956-5e65-4c0b-8d68-d932e789d4a6/pull/0.log" Jul 30 01:48:25.052883 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:25.052859 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb_1503a956-5e65-4c0b-8d68-d932e789d4a6/extract/0.log" Jul 30 01:48:25.167586 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:25.167559 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p_07942485-9a87-436d-8ab9-72797e8e36ae/extract/0.log" Jul 30 01:48:25.173670 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:25.173608 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p_07942485-9a87-436d-8ab9-72797e8e36ae/util/0.log" Jul 30 01:48:25.179565 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:25.179546 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p_07942485-9a87-436d-8ab9-72797e8e36ae/pull/0.log" Jul 30 01:48:25.292282 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:25.292252 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc_721d5439-21ee-46a5-97a8-b34345828dd0/extract/0.log" Jul 30 01:48:25.298089 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:25.298067 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc_721d5439-21ee-46a5-97a8-b34345828dd0/util/0.log" Jul 30 01:48:25.304080 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:25.304062 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc_721d5439-21ee-46a5-97a8-b34345828dd0/pull/0.log" Jul 30 01:48:26.581443 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:26.581417 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76_575ac028-f779-46c9-a84e-2476fee11fe8/istio-proxy/0.log" Jul 30 01:48:26.698288 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:26.698261 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-g2m8j_330c2ba1-c814-4f05-85c9-de6508091b8c/discovery/0.log" Jul 30 01:48:27.030120 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:27.030041 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jffhc_ba299a42-e1c8-4551-a790-da168ca7f972/istio-proxy/1.log" Jul 30 01:48:27.380710 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:27.380678 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67c4d7bd47-tszsf_49165c64-0239-4c5d-b252-218818d09280/router/0.log" Jul 30 01:48:27.834635 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:27.834600 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qcc4q_d5a29493-9c81-4c16-a645-c961b8f1820b/main/0.log" Jul 30 01:48:27.944791 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:27.944767 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-2lgmh_49a1d67b-a216-4fdf-a167-e7218161c8c9/main/0.log" Jul 30 01:48:28.155617 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:28.155535 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-dwmjk_0b629936-5f15-4329-b291-daa5cdff78f3/main/0.log" Jul 30 01:48:33.458367 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:33.458338 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:48:33.561638 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:33.561602 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" limit=1024 Jul 30 01:48:36.321135 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:36.321086 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dj2qr_962582df-c2b5-4b1f-990a-18b7a1c3ccfd/global-pull-secret-syncer/0.log" Jul 30 01:48:36.411817 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:36.411783 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ds2cm_84f859e4-75b2-4f4d-b6af-292c6b5fe003/konnectivity-agent/0.log" Jul 30 01:48:36.545863 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:36.545830 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-247.ec2.internal_f42d3ae68cadbebcf1e1b0f116456906/haproxy/0.log" Jul 30 01:48:41.180235 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:41.180198 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks_2c4ece51-398a-41c3-9cd8-87d4cce68a8b/extract/0.log" Jul 30 01:48:41.201142 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:41.201104 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks_2c4ece51-398a-41c3-9cd8-87d4cce68a8b/util/0.log" Jul 30 01:48:41.223154 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:41.223126 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t77ks_2c4ece51-398a-41c3-9cd8-87d4cce68a8b/pull/0.log" Jul 30 01:48:41.268322 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:41.268288 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb_1503a956-5e65-4c0b-8d68-d932e789d4a6/extract/0.log" Jul 30 01:48:41.289619 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:41.289586 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb_1503a956-5e65-4c0b-8d68-d932e789d4a6/util/0.log" Jul 30 01:48:41.314489 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:41.314446 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bcrkb_1503a956-5e65-4c0b-8d68-d932e789d4a6/pull/0.log" Jul 30 01:48:41.340712 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:41.340683 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p_07942485-9a87-436d-8ab9-72797e8e36ae/extract/0.log" Jul 30 01:48:41.385645 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:41.385616 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p_07942485-9a87-436d-8ab9-72797e8e36ae/util/0.log" Jul 30 01:48:41.408116 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:41.408084 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sgj7p_07942485-9a87-436d-8ab9-72797e8e36ae/pull/0.log" Jul 30 01:48:41.436259 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:41.436176 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc_721d5439-21ee-46a5-97a8-b34345828dd0/extract/0.log" Jul 30 01:48:41.460731 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:41.460702 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc_721d5439-21ee-46a5-97a8-b34345828dd0/util/0.log" Jul 30 01:48:41.491087 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:41.491056 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19msnc_721d5439-21ee-46a5-97a8-b34345828dd0/pull/0.log" Jul 30 01:48:43.911264 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:43.911226 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_35cd7b71-a14b-4317-b4b0-214f58ad92b5/alertmanager/0.log" Jul 30 01:48:43.936669 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:43.936639 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_35cd7b71-a14b-4317-b4b0-214f58ad92b5/config-reloader/0.log" Jul 30 01:48:43.959728 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:43.959700 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_35cd7b71-a14b-4317-b4b0-214f58ad92b5/kube-rbac-proxy-web/0.log" Jul 30 01:48:43.990126 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:43.990093 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_35cd7b71-a14b-4317-b4b0-214f58ad92b5/kube-rbac-proxy/0.log" Jul 30 01:48:44.013320 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.013290 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_35cd7b71-a14b-4317-b4b0-214f58ad92b5/kube-rbac-proxy-metric/0.log" Jul 30 01:48:44.036369 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.036342 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_35cd7b71-a14b-4317-b4b0-214f58ad92b5/prom-label-proxy/0.log" Jul 30 01:48:44.058327 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.058301 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_35cd7b71-a14b-4317-b4b0-214f58ad92b5/init-config-reloader/0.log" Jul 30 01:48:44.125413 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.125379 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-qnlm9_2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de/kube-state-metrics/0.log" Jul 30 01:48:44.144095 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.144069 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-qnlm9_2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de/kube-rbac-proxy-main/0.log" Jul 30 01:48:44.165497 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.165427 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-qnlm9_2c3e8ce9-db86-42a0-a06e-cd2ecc75d6de/kube-rbac-proxy-self/0.log" Jul 30 01:48:44.195630 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.195605 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5d7dd9c977-8bdmd_969cb85c-a22f-4477-a864-75194e9af1da/metrics-server/0.log" Jul 30 01:48:44.224329 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.224303 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-jbr7c_ce77146e-a447-4a19-a1c6-b9fffbac4144/monitoring-plugin/0.log" Jul 30 01:48:44.421595 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.421513 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mt8qv_2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8/node-exporter/0.log" Jul 30 01:48:44.453217 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.453188 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mt8qv_2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8/kube-rbac-proxy/0.log" Jul 30 01:48:44.494201 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.494173 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mt8qv_2d9cec99-d40d-43a2-a4f5-3359a4d0b3a8/init-textfile/0.log" Jul 30 01:48:44.844829 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.844800 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c6755f48-r844c_56a337c5-cf93-41ce-83cb-d41d9b839600/telemeter-client/0.log" Jul 30 01:48:44.865199 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.865173 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c6755f48-r844c_56a337c5-cf93-41ce-83cb-d41d9b839600/reload/0.log" Jul 30 01:48:44.885328 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.885299 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c6755f48-r844c_56a337c5-cf93-41ce-83cb-d41d9b839600/kube-rbac-proxy/0.log" Jul 30 01:48:44.916194 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.916168 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b977d57-mrbbw_2a889c73-df8d-4baf-bb64-8bc313d57684/thanos-query/0.log" Jul 30 01:48:44.936457 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.936426 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b977d57-mrbbw_2a889c73-df8d-4baf-bb64-8bc313d57684/kube-rbac-proxy-web/0.log" Jul 30 01:48:44.957715 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.957689 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b977d57-mrbbw_2a889c73-df8d-4baf-bb64-8bc313d57684/kube-rbac-proxy/0.log" Jul 30 01:48:44.977913 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.977867 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b977d57-mrbbw_2a889c73-df8d-4baf-bb64-8bc313d57684/prom-label-proxy/0.log" Jul 30 01:48:44.998656 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:44.998628 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b977d57-mrbbw_2a889c73-df8d-4baf-bb64-8bc313d57684/kube-rbac-proxy-rules/0.log" Jul 30 01:48:45.020044 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:45.020018 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b977d57-mrbbw_2a889c73-df8d-4baf-bb64-8bc313d57684/kube-rbac-proxy-metrics/0.log" Jul 30 01:48:46.427387 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.427347 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j"] Jul 30 01:48:46.428112 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.427724 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="663cfac9-7b75-49cd-8244-884ca6ea32bd" containerName="revoke-keys" Jul 30 01:48:46.428112 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.427735 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="663cfac9-7b75-49cd-8244-884ca6ea32bd" containerName="revoke-keys" Jul 30 01:48:46.428112 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.427746 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c18af419-ea76-4774-b5b2-608b96926159" containerName="istio-proxy" Jul 30 01:48:46.428112 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.427752 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c18af419-ea76-4774-b5b2-608b96926159" containerName="istio-proxy" Jul 30 01:48:46.428112 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.427764 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c18af419-ea76-4774-b5b2-608b96926159" containerName="istio-proxy" Jul 30 01:48:46.428112 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.427772 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c18af419-ea76-4774-b5b2-608b96926159" containerName="istio-proxy" Jul 30 01:48:46.428112 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.427826 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="663cfac9-7b75-49cd-8244-884ca6ea32bd" containerName="revoke-keys" Jul 30 01:48:46.428112 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.427835 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c18af419-ea76-4774-b5b2-608b96926159" containerName="istio-proxy" Jul 30 01:48:46.431153 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.431133 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.433346 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.433320 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kwqhr\"/\"kube-root-ca.crt\"" Jul 30 01:48:46.433470 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.433325 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kwqhr\"/\"openshift-service-ca.crt\"" Jul 30 01:48:46.434099 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.434082 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kwqhr\"/\"default-dockercfg-hsst4\"" Jul 30 01:48:46.439143 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.438609 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j"] Jul 30 01:48:46.587144 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.587105 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhtj6\" (UniqueName: \"kubernetes.io/projected/2d7f9e92-2be7-41d2-acc0-06278962120d-kube-api-access-zhtj6\") pod \"perf-node-gather-daemonset-bl85j\" (UID: \"2d7f9e92-2be7-41d2-acc0-06278962120d\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.587330 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.587179 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d7f9e92-2be7-41d2-acc0-06278962120d-podres\") pod \"perf-node-gather-daemonset-bl85j\" (UID: \"2d7f9e92-2be7-41d2-acc0-06278962120d\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.587330 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.587223 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d7f9e92-2be7-41d2-acc0-06278962120d-sys\") pod \"perf-node-gather-daemonset-bl85j\" (UID: \"2d7f9e92-2be7-41d2-acc0-06278962120d\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.587330 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.587240 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d7f9e92-2be7-41d2-acc0-06278962120d-lib-modules\") pod \"perf-node-gather-daemonset-bl85j\" (UID: \"2d7f9e92-2be7-41d2-acc0-06278962120d\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.587330 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.587314 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d7f9e92-2be7-41d2-acc0-06278962120d-proc\") pod \"perf-node-gather-daemonset-bl85j\" (UID: \"2d7f9e92-2be7-41d2-acc0-06278962120d\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.688726 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.688633 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d7f9e92-2be7-41d2-acc0-06278962120d-proc\") pod \"perf-node-gather-daemonset-bl85j\" (UID: \"2d7f9e92-2be7-41d2-acc0-06278962120d\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.688726 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.688682 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhtj6\" (UniqueName: \"kubernetes.io/projected/2d7f9e92-2be7-41d2-acc0-06278962120d-kube-api-access-zhtj6\") pod \"perf-node-gather-daemonset-bl85j\" (UID: \"2d7f9e92-2be7-41d2-acc0-06278962120d\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.688967 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.688740 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d7f9e92-2be7-41d2-acc0-06278962120d-podres\") pod \"perf-node-gather-daemonset-bl85j\" (UID: \"2d7f9e92-2be7-41d2-acc0-06278962120d\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.688967 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.688764 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d7f9e92-2be7-41d2-acc0-06278962120d-sys\") pod \"perf-node-gather-daemonset-bl85j\" (UID: \"2d7f9e92-2be7-41d2-acc0-06278962120d\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.688967 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.688784 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d7f9e92-2be7-41d2-acc0-06278962120d-lib-modules\") pod \"perf-node-gather-daemonset-bl85j\" (UID: \"2d7f9e92-2be7-41d2-acc0-06278962120d\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.688967 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.688772 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d7f9e92-2be7-41d2-acc0-06278962120d-proc\") pod \"perf-node-gather-daemonset-bl85j\" (UID: \"2d7f9e92-2be7-41d2-acc0-06278962120d\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.688967 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.688870 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d7f9e92-2be7-41d2-acc0-06278962120d-sys\") pod \"perf-node-gather-daemonset-bl85j\" (UID: \"2d7f9e92-2be7-41d2-acc0-06278962120d\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.688967 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.688934 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d7f9e92-2be7-41d2-acc0-06278962120d-podres\") pod \"perf-node-gather-daemonset-bl85j\" (UID: \"2d7f9e92-2be7-41d2-acc0-06278962120d\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.688967 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.688944 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d7f9e92-2be7-41d2-acc0-06278962120d-lib-modules\") pod \"perf-node-gather-daemonset-bl85j\" (UID: \"2d7f9e92-2be7-41d2-acc0-06278962120d\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.696434 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.696406 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhtj6\" (UniqueName: \"kubernetes.io/projected/2d7f9e92-2be7-41d2-acc0-06278962120d-kube-api-access-zhtj6\") pod \"perf-node-gather-daemonset-bl85j\" (UID: \"2d7f9e92-2be7-41d2-acc0-06278962120d\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.742267 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.742226 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:46.888017 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:46.887990 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j"] Jul 30 01:48:46.890056 ip-10-0-136-247 kubenswrapper[2564]: W0730 01:48:46.890028 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2d7f9e92_2be7_41d2_acc0_06278962120d.slice/crio-3677cfc98418aed0311b24f99aff89b7f9397621da3262516cf2db00d15005eb WatchSource:0}: Error finding container 3677cfc98418aed0311b24f99aff89b7f9397621da3262516cf2db00d15005eb: Status 404 returned error can't find the container with id 3677cfc98418aed0311b24f99aff89b7f9397621da3262516cf2db00d15005eb Jul 30 01:48:47.388155 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:47.388124 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-nc5kn_7f5a55fe-584b-402f-8850-758a3367d187/download-server/0.log" Jul 30 01:48:47.781682 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:47.781587 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" event={"ID":"2d7f9e92-2be7-41d2-acc0-06278962120d","Type":"ContainerStarted","Data":"6af5fcae80e9d45dcf2d5dcc9605246c89793bc616738f46f2d2c043c1f8c2eb"} Jul 30 01:48:47.781682 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:47.781669 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" event={"ID":"2d7f9e92-2be7-41d2-acc0-06278962120d","Type":"ContainerStarted","Data":"3677cfc98418aed0311b24f99aff89b7f9397621da3262516cf2db00d15005eb"} Jul 30 01:48:47.782119 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:47.781694 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:47.797169 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:47.797098 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" podStartSLOduration=1.797075806 podStartE2EDuration="1.797075806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 01:48:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:48:47.796267644 +0000 UTC m=+3930.597224755" watchObservedRunningTime="2026-07-30 01:48:47.797075806 +0000 UTC m=+3930.598032912" Jul 30 01:48:48.973520 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:48.973485 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8cxk4_5c59f8b2-e65b-4f30-93bb-16a06e3d5f45/dns/0.log" Jul 30 01:48:48.998844 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:48.998815 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8cxk4_5c59f8b2-e65b-4f30-93bb-16a06e3d5f45/kube-rbac-proxy/0.log" Jul 30 01:48:49.109012 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:49.108986 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7xg42_61e4710e-5c0b-475d-80d4-db9f1cc78e27/dns-node-resolver/0.log" Jul 30 01:48:49.821432 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:49.821397 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xqfqf_aef50669-7ed3-432c-ad80-62254f6a5ac7/node-ca/0.log" Jul 30 01:48:50.783228 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:50.783197 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76_575ac028-f779-46c9-a84e-2476fee11fe8/istio-proxy/0.log" Jul 30 01:48:50.933932 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:50.933886 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-g2m8j_330c2ba1-c814-4f05-85c9-de6508091b8c/discovery/0.log" Jul 30 01:48:51.012826 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:51.012776 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jffhc_ba299a42-e1c8-4551-a790-da168ca7f972/istio-proxy/0.log" Jul 30 01:48:51.042822 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:51.042725 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jffhc_ba299a42-e1c8-4551-a790-da168ca7f972/istio-proxy/1.log" Jul 30 01:48:51.148813 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:51.148780 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67c4d7bd47-tszsf_49165c64-0239-4c5d-b252-218818d09280/router/0.log" Jul 30 01:48:51.907468 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:51.907435 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4t9fb_7d1f5f34-14d3-4458-9c22-f6b33bcea80e/serve-healthcheck-canary/0.log" Jul 30 01:48:52.445772 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:52.445740 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bg6hz_610a5275-fbb6-4501-bc53-a31f61798db0/kube-rbac-proxy/0.log" Jul 30 01:48:52.468924 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:52.468863 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bg6hz_610a5275-fbb6-4501-bc53-a31f61798db0/exporter/0.log" Jul 30 01:48:52.489947 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:52.489882 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bg6hz_610a5275-fbb6-4501-bc53-a31f61798db0/extractor/0.log" Jul 30 01:48:53.798724 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:53.798696 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-bl85j" Jul 30 01:48:55.760106 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:55.760024 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b4b6684-htpnw_b2dfe990-5238-4ad7-8c27-43dfde9eda24/manager/0.log" Jul 30 01:48:55.780208 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:55.780184 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b4b6684-htpnw_b2dfe990-5238-4ad7-8c27-43dfde9eda24/copy-manifests/0.log" Jul 30 01:48:57.090352 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:57.090323 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76_575ac028-f779-46c9-a84e-2476fee11fe8/istio-proxy/0.log" Jul 30 01:48:57.129806 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:57.129773 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6d674c9db-9qm8x_f8c7cf3b-633e-4d2e-9586-e4995d2cc743/manager/0.log" Jul 30 01:48:57.238929 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:57.238879 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-g2m8j_330c2ba1-c814-4f05-85c9-de6508091b8c/discovery/0.log" Jul 30 01:48:57.300824 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:57.300793 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jffhc_ba299a42-e1c8-4551-a790-da168ca7f972/istio-proxy/0.log" Jul 30 01:48:57.326854 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:57.326806 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jffhc_ba299a42-e1c8-4551-a790-da168ca7f972/istio-proxy/1.log" Jul 30 01:48:57.422149 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:57.422049 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67c4d7bd47-tszsf_49165c64-0239-4c5d-b252-218818d09280/router/0.log" Jul 30 01:48:58.820388 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:58.820359 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-lpl76_575ac028-f779-46c9-a84e-2476fee11fe8/istio-proxy/0.log" Jul 30 01:48:58.963672 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:58.963641 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-g2m8j_330c2ba1-c814-4f05-85c9-de6508091b8c/discovery/0.log" Jul 30 01:48:59.009810 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:59.009719 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jffhc_ba299a42-e1c8-4551-a790-da168ca7f972/istio-proxy/0.log" Jul 30 01:48:59.035882 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:59.035832 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jffhc_ba299a42-e1c8-4551-a790-da168ca7f972/istio-proxy/1.log" Jul 30 01:48:59.114859 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:48:59.114827 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67c4d7bd47-tszsf_49165c64-0239-4c5d-b252-218818d09280/router/0.log" Jul 30 01:49:03.619535 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:49:03.619510 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pjkjg_0404fd97-1663-44e4-bf04-6a588423f8ed/kube-multus-additional-cni-plugins/0.log" Jul 30 01:49:03.638957 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:49:03.638926 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pjkjg_0404fd97-1663-44e4-bf04-6a588423f8ed/egress-router-binary-copy/0.log" Jul 30 01:49:03.658581 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:49:03.658555 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pjkjg_0404fd97-1663-44e4-bf04-6a588423f8ed/cni-plugins/0.log" Jul 30 01:49:03.678483 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:49:03.678443 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pjkjg_0404fd97-1663-44e4-bf04-6a588423f8ed/bond-cni-plugin/0.log" Jul 30 01:49:03.707026 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:49:03.706992 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pjkjg_0404fd97-1663-44e4-bf04-6a588423f8ed/routeoverride-cni/0.log" Jul 30 01:49:03.727265 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:49:03.727236 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pjkjg_0404fd97-1663-44e4-bf04-6a588423f8ed/whereabouts-cni-bincopy/0.log" Jul 30 01:49:03.746586 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:49:03.746558 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pjkjg_0404fd97-1663-44e4-bf04-6a588423f8ed/whereabouts-cni/0.log" Jul 30 01:49:03.991766 ip-10-0-136-247 kubenswrapper[2564]: I0730 01:49:03.991688 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tvd2g_1fca91ee-9cbd-4da1-b318-96ab379123e0/kube-multus/0.log"