Jul 31 22:55:18.107071 ip-10-0-131-148 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 31 22:55:18.107079 ip-10-0-131-148 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 31 22:55:18.107086 ip-10-0-131-148 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 31 22:55:18.107305 ip-10-0-131-148 systemd[1]: Failed to start Kubernetes Kubelet. Jul 31 22:55:28.298271 ip-10-0-131-148 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 31 22:55:28.298286 ip-10-0-131-148 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot d61c500717b74124a59c7df6122c42a5 -- Jul 31 22:57:23.401801 ip-10-0-131-148 systemd[1]: Starting Kubernetes Kubelet... Jul 31 22:57:23.846086 ip-10-0-131-148 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 31 22:57:23.846086 ip-10-0-131-148 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 31 22:57:23.846086 ip-10-0-131-148 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 31 22:57:23.846086 ip-10-0-131-148 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 31 22:57:23.846086 ip-10-0-131-148 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 31 22:57:23.848362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.848159 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 31 22:57:23.854936 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854904 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 22:57:23.854936 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854931 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 22:57:23.854936 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854937 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 22:57:23.854936 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854944 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854950 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854955 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854959 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854963 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854967 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854971 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854975 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854979 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854982 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854986 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854992 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.854998 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855002 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855006 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855010 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855019 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855024 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855028 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 22:57:23.855257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855034 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855042 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855047 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855052 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855057 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855062 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855066 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855070 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855074 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855079 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855083 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855087 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855091 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855095 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855101 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855107 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855112 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855117 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855121 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855126 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 22:57:23.856183 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855130 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855135 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855139 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855144 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855148 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855153 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855158 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855165 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855182 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855188 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855193 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855197 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855202 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855207 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855213 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855217 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855222 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855226 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855231 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 22:57:23.857095 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855236 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855240 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855245 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855249 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855254 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855261 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855265 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855270 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855275 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855281 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855286 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855290 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855295 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855300 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855304 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855309 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855314 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855318 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855322 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855327 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 22:57:23.857985 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855331 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855335 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855349 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855354 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855359 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855364 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855368 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855372 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855377 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855382 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855387 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855392 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855396 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855401 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855405 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855409 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855414 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.855419 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856794 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856808 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 22:57:23.858946 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856817 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856822 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856828 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856833 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856840 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856847 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856854 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856860 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856865 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856869 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856875 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856879 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856884 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856890 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856894 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856900 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856905 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856909 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856914 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 22:57:23.859878 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856918 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856923 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856927 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856932 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856936 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856941 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856946 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856950 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856955 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856959 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856963 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856968 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856974 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856981 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856986 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856990 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856995 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.856999 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857004 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 22:57:23.860561 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857008 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857013 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857017 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857022 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857027 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857031 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857036 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857041 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857046 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857051 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857057 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857061 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857066 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857071 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857075 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857080 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857084 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857089 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857094 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 22:57:23.861108 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857099 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857106 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857111 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857116 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857121 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857126 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857132 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857140 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857145 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857150 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857155 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857160 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857164 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857168 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857173 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857177 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857182 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857186 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857191 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857195 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 22:57:23.861663 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857199 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857204 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857208 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857212 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857218 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857225 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857233 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857238 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857242 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857247 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857251 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857256 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857260 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857265 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857269 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857274 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857278 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857282 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857287 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.857294 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 22:57:23.862192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858022 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858032 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858037 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858042 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858047 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858054 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858061 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858066 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858071 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858076 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858081 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858085 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858090 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858095 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858100 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858104 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858110 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858114 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858119 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 22:57:23.862755 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858125 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858130 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858134 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858139 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858143 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858148 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858152 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858157 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858162 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858166 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858171 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858178 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858185 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858189 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858194 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858198 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858202 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858207 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858211 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858216 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 22:57:23.863406 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858220 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858225 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858229 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858234 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858238 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858243 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858247 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858251 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858256 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858261 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858266 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858271 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858275 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858279 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858284 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858288 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858292 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858297 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858301 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 22:57:23.864189 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858305 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858310 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858314 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858322 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858326 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858332 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858336 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858341 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858346 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858350 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858357 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858362 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858366 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858371 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858375 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858379 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858384 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858388 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858393 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 22:57:23.864796 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858399 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858403 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858408 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858413 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858417 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858422 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858427 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858431 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858437 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858442 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858447 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858451 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858456 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858463 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858469 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858474 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858481 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858486 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858492 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858497 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 22:57:23.865341 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858502 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.858506 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858635 2561 flags.go:64] FLAG: --address="0.0.0.0" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858650 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858661 2561 flags.go:64] FLAG: --anonymous-auth="true" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858668 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858676 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858681 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858688 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858696 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858701 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858707 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858713 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858719 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858724 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858730 2561 flags.go:64] FLAG: --cgroup-root="" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858734 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858740 2561 flags.go:64] FLAG: --client-ca-file="" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858746 2561 flags.go:64] FLAG: --cloud-provider="external" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858752 2561 flags.go:64] FLAG: --cluster-dns="[]" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858759 2561 flags.go:64] FLAG: --cluster-domain="" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858764 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858769 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858775 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 31 22:57:23.866011 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858781 2561 flags.go:64] FLAG: --container-log-max-files="5" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858788 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858793 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858799 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858807 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858812 2561 flags.go:64] FLAG: --contention-profiling="false" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858819 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858825 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858831 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858836 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858843 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858848 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858854 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858858 2561 flags.go:64] FLAG: --enable-load-reader="false" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858864 2561 flags.go:64] FLAG: --enable-server="true" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858869 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858877 2561 flags.go:64] FLAG: --event-burst="100" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858883 2561 flags.go:64] FLAG: --event-qps="50" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858888 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858893 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858898 2561 flags.go:64] FLAG: --eviction-hard="" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858905 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858910 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858916 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858921 2561 flags.go:64] FLAG: --eviction-soft="" Jul 31 22:57:23.866712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858926 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858931 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858938 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858943 2561 flags.go:64] FLAG: --experimental-mounter-path="" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858948 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858953 2561 flags.go:64] FLAG: --fail-swap-on="true" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858958 2561 flags.go:64] FLAG: --feature-gates="" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858964 2561 flags.go:64] FLAG: --file-check-frequency="20s" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858970 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858975 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858980 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858988 2561 flags.go:64] FLAG: --healthz-port="10248" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.858993 2561 flags.go:64] FLAG: --help="false" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859000 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-131-148.ec2.internal" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859005 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859010 2561 flags.go:64] FLAG: --http-check-frequency="20s" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859015 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859021 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859027 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859032 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859037 2561 flags.go:64] FLAG: --image-service-endpoint="" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859042 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859047 2561 flags.go:64] FLAG: --kube-api-burst="100" Jul 31 22:57:23.867379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859052 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859058 2561 flags.go:64] FLAG: --kube-api-qps="50" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859063 2561 flags.go:64] FLAG: --kube-reserved="" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859068 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859073 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859079 2561 flags.go:64] FLAG: --kubelet-cgroups="" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859084 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859089 2561 flags.go:64] FLAG: --lock-file="" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859095 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859100 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859105 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859115 2561 flags.go:64] FLAG: --log-json-split-stream="false" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859121 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859126 2561 flags.go:64] FLAG: --log-text-split-stream="false" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859132 2561 flags.go:64] FLAG: --logging-format="text" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859137 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859143 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859148 2561 flags.go:64] FLAG: --manifest-url="" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859153 2561 flags.go:64] FLAG: --manifest-url-header="" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859160 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859168 2561 flags.go:64] FLAG: --max-open-files="1000000" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859176 2561 flags.go:64] FLAG: --max-pods="110" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859181 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859187 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859192 2561 flags.go:64] FLAG: --memory-manager-policy="None" Jul 31 22:57:23.868043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859197 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859202 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859208 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859213 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859226 2561 flags.go:64] FLAG: --node-status-max-images="50" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859232 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859237 2561 flags.go:64] FLAG: --oom-score-adj="-999" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859242 2561 flags.go:64] FLAG: --pod-cidr="" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859247 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859257 2561 flags.go:64] FLAG: --pod-manifest-path="" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859262 2561 flags.go:64] FLAG: --pod-max-pids="-1" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859267 2561 flags.go:64] FLAG: --pods-per-core="0" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859272 2561 flags.go:64] FLAG: --port="10250" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859278 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859283 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f39f6d434930636d" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859289 2561 flags.go:64] FLAG: --qos-reserved="" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859294 2561 flags.go:64] FLAG: --read-only-port="10255" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859299 2561 flags.go:64] FLAG: --register-node="true" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859305 2561 flags.go:64] FLAG: --register-with-taints="" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859311 2561 flags.go:64] FLAG: --registry-burst="10" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859317 2561 flags.go:64] FLAG: --registry-qps="5" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859322 2561 flags.go:64] FLAG: --reserved-cpus="" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859328 2561 flags.go:64] FLAG: --reserved-memory="" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859334 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859339 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 31 22:57:23.868846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859345 2561 flags.go:64] FLAG: --rotate-certificates="false" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859350 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859356 2561 flags.go:64] FLAG: --runonce="false" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859363 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859368 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859374 2561 flags.go:64] FLAG: --seccomp-default="false" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859379 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859383 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859389 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859394 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859400 2561 flags.go:64] FLAG: --storage-driver-password="root" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859405 2561 flags.go:64] FLAG: --storage-driver-secure="false" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859410 2561 flags.go:64] FLAG: --storage-driver-table="stats" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859416 2561 flags.go:64] FLAG: --storage-driver-user="root" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859421 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859426 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859431 2561 flags.go:64] FLAG: --system-cgroups="" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859436 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859446 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859450 2561 flags.go:64] FLAG: --tls-cert-file="" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859456 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859463 2561 flags.go:64] FLAG: --tls-min-version="" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859467 2561 flags.go:64] FLAG: --tls-private-key-file="" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859472 2561 flags.go:64] FLAG: --topology-manager-policy="none" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859478 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 31 22:57:23.869589 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859483 2561 flags.go:64] FLAG: --topology-manager-scope="container" Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859488 2561 flags.go:64] FLAG: --v="2" Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859495 2561 flags.go:64] FLAG: --version="false" Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859510 2561 flags.go:64] FLAG: --vmodule="" Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859517 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.859523 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859748 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859759 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859764 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859769 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859776 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859780 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859785 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859789 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859794 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859799 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859804 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859808 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859813 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859818 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859822 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 22:57:23.870272 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859827 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859831 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859836 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859841 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859848 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859854 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859860 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859865 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859870 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859875 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859879 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859883 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859887 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859891 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859896 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859905 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859910 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859915 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859919 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 22:57:23.870862 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859924 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859928 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859934 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859941 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859946 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859950 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859955 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859960 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859965 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859969 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859974 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859980 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859985 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859989 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859994 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.859998 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860003 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860007 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860012 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860017 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 22:57:23.871365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860021 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860026 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860030 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860035 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860040 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860044 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860049 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860054 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860059 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860065 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860069 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860074 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860078 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860083 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860089 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860097 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860101 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860106 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860111 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860115 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 31 22:57:23.871949 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860120 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860124 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860129 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860133 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860138 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860143 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860147 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860155 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860160 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860165 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860171 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860175 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860180 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860184 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860189 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860194 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860198 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860202 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860207 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 22:57:23.872481 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860212 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 22:57:23.873118 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860216 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 22:57:23.873118 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860221 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 22:57:23.873118 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860227 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 22:57:23.873118 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860232 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 22:57:23.873118 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.860236 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 22:57:23.873118 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.862868 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 31 22:57:23.873118 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.870423 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 31 22:57:23.873118 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.870440 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 31 22:57:23.873118 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.871208 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 31 22:57:23.873118 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.871217 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 31 22:57:23.873118 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871281 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 22:57:23.873118 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871286 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 22:57:23.873118 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871290 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 22:57:23.873118 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871294 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871297 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871301 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871304 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871306 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871309 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871312 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871315 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871318 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871321 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871324 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871327 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871330 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871332 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871335 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871338 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871341 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871343 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871346 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 22:57:23.873497 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871349 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871352 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871355 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871359 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871362 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871365 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871368 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871372 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871376 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871379 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871382 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871385 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871388 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871390 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871395 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871399 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871403 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871407 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871410 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871412 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 31 22:57:23.874023 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871415 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871418 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871421 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871424 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871427 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871429 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871432 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871435 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871438 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871440 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871443 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871446 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871449 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871452 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871455 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871460 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871464 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871467 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871471 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871474 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 22:57:23.874565 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871477 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871480 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871483 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871488 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871491 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871495 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871498 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871501 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871505 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871508 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871511 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871514 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871517 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871519 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871522 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871536 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871539 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871542 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871545 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 22:57:23.875167 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871548 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871552 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871554 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871557 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871560 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871563 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871566 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871568 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871571 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871576 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871578 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871582 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871586 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871590 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871595 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871599 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871602 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 22:57:23.875710 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871605 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 22:57:23.876152 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.871611 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 31 22:57:23.876152 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871771 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 22:57:23.876152 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871777 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 22:57:23.876152 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871781 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 22:57:23.876152 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871784 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 22:57:23.876152 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871787 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 22:57:23.876152 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871790 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 22:57:23.876152 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871793 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 22:57:23.876152 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871796 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 22:57:23.876152 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871799 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 22:57:23.876152 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871802 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 22:57:23.876152 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871806 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 22:57:23.876152 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871809 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 22:57:23.876152 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871811 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 22:57:23.876152 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871814 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871817 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871820 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871824 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871828 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871831 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871834 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871838 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871841 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871844 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871858 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871862 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871865 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871868 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871871 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871874 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871877 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871880 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871882 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 22:57:23.876558 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871885 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871889 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871894 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871897 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871900 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871902 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871905 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871908 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871912 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871916 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871919 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871923 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871926 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871929 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871932 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871935 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871938 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871940 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871943 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 22:57:23.877106 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871946 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871949 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871952 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871955 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871958 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871962 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871966 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871969 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871971 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871974 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871977 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871980 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871983 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871986 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871988 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871991 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871994 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871997 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.871999 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872002 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 22:57:23.877629 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872005 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872008 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872010 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872013 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872016 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872019 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872022 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872024 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872027 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872030 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872033 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872035 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872038 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872041 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872043 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872046 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872049 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872052 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872055 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872058 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 22:57:23.878160 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872061 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 22:57:23.878715 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872064 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 22:57:23.878715 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872067 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 22:57:23.878715 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872069 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 22:57:23.878715 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872072 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 22:57:23.878715 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872075 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 22:57:23.878715 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872077 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 22:57:23.878715 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:23.872080 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 22:57:23.878715 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.872085 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 31 22:57:23.878715 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.872219 2561 server.go:962] "Client rotation is on, will bootstrap in background" Jul 31 22:57:23.878715 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.874526 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 31 22:57:23.878715 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.876258 2561 server.go:1019] "Starting client certificate rotation" Jul 31 22:57:23.878715 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.876369 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 31 22:57:23.878715 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.876401 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 31 22:57:23.879198 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.879181 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 31 22:57:23.880819 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.880798 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 31 22:57:23.893295 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.893268 2561 log.go:25] "Validated CRI v1 runtime API" Jul 31 22:57:23.895788 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.895770 2561 log.go:25] "Validated CRI v1 image API" Jul 31 22:57:23.896800 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.896782 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 31 22:57:23.900290 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.900260 2561 fs.go:135] Filesystem UUIDs: map[141d47c6-8054-4ffa-9648-ea7a35897675:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ca0fbe56-97ce-4bd4-a093-3c2e2a56b62e:/dev/nvme0n1p3] Jul 31 22:57:23.900395 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.900289 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 31 22:57:23.906390 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.906271 2561 manager.go:216] Machine: {Timestamp:2026-07-31 22:57:23.904550886 +0000 UTC m=+0.376995063 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107772 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e0fdd28fbb21e512d98c4c0be8a25 SystemUUID:ec2e0fdd-28fb-b21e-512d-98c4c0be8a25 BootID:d61c5007-17b7-4124-a59c-7df6122c42a5 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c1:37:fd:18:cb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c1:37:fd:18:cb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:b3:13:de:8a:0f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 31 22:57:23.906390 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.906384 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 31 22:57:23.906508 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.906474 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 31 22:57:23.909737 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.909702 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 31 22:57:23.909887 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.909740 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-148.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 31 22:57:23.910203 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.910185 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 31 22:57:23.911305 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.911293 2561 topology_manager.go:138] "Creating topology manager with none policy" Jul 31 22:57:23.911352 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.911308 2561 container_manager_linux.go:309] "Creating device plugin manager" Jul 31 22:57:23.911352 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.911316 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 31 22:57:23.911945 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.911934 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 31 22:57:23.912786 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.912748 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 31 22:57:23.916379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.916366 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 31 22:57:23.916518 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.916508 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 31 22:57:23.916550 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.916542 2561 kubelet.go:486] "Attempting to sync node with API server" Jul 31 22:57:23.916591 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.916556 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 31 22:57:23.916591 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.916573 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 31 22:57:23.916670 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.916596 2561 kubelet.go:398] "Adding apiserver pod source" Jul 31 22:57:23.916670 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.916608 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 31 22:57:23.917012 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.916998 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 31 22:57:23.917063 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.917017 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 31 22:57:23.919652 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.919632 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 31 22:57:23.921182 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.921168 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 31 22:57:23.921259 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.921199 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 31 22:57:23.922320 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.922306 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 31 22:57:23.922373 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.922326 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 31 22:57:23.922373 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.922333 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 31 22:57:23.922373 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.922338 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 31 22:57:23.922373 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.922348 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 31 22:57:23.922373 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.922354 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 31 22:57:23.922373 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.922363 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 31 22:57:23.922373 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.922369 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 31 22:57:23.922565 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.922379 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 31 22:57:23.922565 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.922386 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 31 22:57:23.922565 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.922399 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 31 22:57:23.922565 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.922405 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 31 22:57:23.923240 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.923229 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 31 22:57:23.923240 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.923240 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 31 22:57:23.926638 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.926618 2561 server.go:1268] "Started kubelet" Jul 31 22:57:23.926738 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.926681 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 31 22:57:23.926843 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.926787 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 31 22:57:23.926907 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.926888 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 31 22:57:23.927565 ip-10-0-131-148 systemd[1]: Started Kubernetes Kubelet. Jul 31 22:57:23.927712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.927699 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-148.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 31 22:57:23.928330 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.928074 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 31 22:57:23.928434 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:23.928402 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 31 22:57:23.928571 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:23.928460 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-148.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 31 22:57:23.929461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.929444 2561 server.go:310] "Adding debug handlers to kubelet server" Jul 31 22:57:23.931921 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:23.931902 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 31 22:57:23.933763 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:23.932475 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-148.ec2.internal.18c781efbbf8c242 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-148.ec2.internal,UID:ip-10-0-131-148.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-148.ec2.internal,},FirstTimestamp:2026-07-31 22:57:23.926499906 +0000 UTC m=+0.398944083,LastTimestamp:2026-07-31 22:57:23.926499906 +0000 UTC m=+0.398944083,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-148.ec2.internal,}" Jul 31 22:57:23.935408 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.935371 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 31 22:57:23.935490 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.935382 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 31 22:57:23.935558 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:23.935525 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 31 22:57:23.936074 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.936057 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 31 22:57:23.936074 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.936066 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 31 22:57:23.936473 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.936449 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 31 22:57:23.936645 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.936630 2561 reconstruct.go:97] "Volume reconstruction finished" Jul 31 22:57:23.936645 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.936645 2561 reconciler.go:29] "Reconciler: start to sync state" Jul 31 22:57:23.936774 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.936750 2561 factory.go:55] Registering systemd factory Jul 31 22:57:23.936774 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.936768 2561 factory.go:223] Registration of the systemd container factory successfully Jul 31 22:57:23.937608 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:23.937580 2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-148.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 31 22:57:23.938438 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.938423 2561 factory.go:153] Registering CRI-O factory Jul 31 22:57:23.938568 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.938557 2561 factory.go:223] Registration of the crio container factory successfully Jul 31 22:57:23.938692 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.938681 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 31 22:57:23.938793 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.938784 2561 factory.go:103] Registering Raw factory Jul 31 22:57:23.938864 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.938857 2561 manager.go:1196] Started watching for new ooms in manager Jul 31 22:57:23.939580 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.939430 2561 manager.go:349] Starting recovery of all containers Jul 31 22:57:23.945323 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.945297 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-88vpt" Jul 31 22:57:23.950136 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:23.950114 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 31 22:57:23.950935 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.950817 2561 manager.go:354] Recovery completed Jul 31 22:57:23.955399 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.955385 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 22:57:23.957073 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.957056 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-88vpt" Jul 31 22:57:23.957537 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.957521 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientMemory" Jul 31 22:57:23.957629 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.957549 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 22:57:23.957629 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.957561 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientPID" Jul 31 22:57:23.958300 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.958278 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 31 22:57:23.958300 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.958296 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 31 22:57:23.958426 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.958312 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 22:57:23.958426 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.958316 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 31 22:57:23.959115 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.959096 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientMemory" Jul 31 22:57:23.959201 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.959131 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 22:57:23.959201 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.959147 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientPID" Jul 31 22:57:23.959625 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.959606 2561 policy_none.go:49] "None policy: Start" Jul 31 22:57:23.959625 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.959627 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 31 22:57:23.959728 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.959638 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 31 22:57:23.960815 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.960802 2561 policy_none.go:47] "Start" Jul 31 22:57:23.997453 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.997245 2561 manager.go:337] "Starting Device Plugin manager" Jul 31 22:57:23.997606 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:23.997482 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 31 22:57:23.997606 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.997498 2561 server.go:87] "Starting device plugin registration server" Jul 31 22:57:23.997848 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.997833 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 31 22:57:23.997894 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.997850 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 31 22:57:23.997970 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.997955 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 31 22:57:23.998066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.998049 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 31 22:57:23.998066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:23.998063 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 31 22:57:23.998614 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:23.998573 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 31 22:57:23.998678 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:23.998643 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-148.ec2.internal\" not found" Jul 31 22:57:24.102292 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.102258 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 22:57:24.105761 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.103600 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientMemory" Jul 31 22:57:24.105761 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.103634 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 22:57:24.105761 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.103646 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientPID" Jul 31 22:57:24.105761 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.103673 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.112741 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.112722 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.112820 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:24.112752 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-148.ec2.internal\": node \"ip-10-0-131-148.ec2.internal\" not found" Jul 31 22:57:24.124613 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.124574 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 31 22:57:24.125828 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.125808 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 31 22:57:24.125879 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.125861 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 31 22:57:24.125924 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.125888 2561 kubelet.go:2448] "Starting kubelet main sync loop" Jul 31 22:57:24.125964 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:24.125933 2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 31 22:57:24.129195 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.129172 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 22:57:24.136236 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:24.136207 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 31 22:57:24.227237 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.227117 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal"] Jul 31 22:57:24.227237 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.227229 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 22:57:24.232102 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.232084 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientMemory" Jul 31 22:57:24.232199 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.232118 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 22:57:24.232199 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.232131 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientPID" Jul 31 22:57:24.234487 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.234471 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 22:57:24.234614 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.234597 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.234664 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.234635 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 22:57:24.235253 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.235236 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientMemory" Jul 31 22:57:24.235333 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.235266 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 22:57:24.235333 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.235277 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientPID" Jul 31 22:57:24.235333 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.235237 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientMemory" Jul 31 22:57:24.235432 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.235353 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 22:57:24.235432 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.235367 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientPID" Jul 31 22:57:24.236366 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:24.236347 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 31 22:57:24.237734 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.237712 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/53f1b9e34b3406032d3a75269838c9d7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal\" (UID: \"53f1b9e34b3406032d3a75269838c9d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.237734 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.237731 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.237847 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.237750 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53f1b9e34b3406032d3a75269838c9d7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal\" (UID: \"53f1b9e34b3406032d3a75269838c9d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.237847 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.237763 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 22:57:24.237847 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.237776 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a98b158e97d3e00d1e6f3fd6fce59404-config\") pod \"kube-apiserver-proxy-ip-10-0-131-148.ec2.internal\" (UID: \"a98b158e97d3e00d1e6f3fd6fce59404\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.238524 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.238497 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientMemory" Jul 31 22:57:24.238639 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.238534 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 22:57:24.238639 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.238546 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientPID" Jul 31 22:57:24.261500 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:24.261471 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-148.ec2.internal\" not found" node="ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.266263 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:24.266241 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-148.ec2.internal\" not found" node="ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.337273 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:24.337239 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 31 22:57:24.338453 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.338433 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/53f1b9e34b3406032d3a75269838c9d7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal\" (UID: \"53f1b9e34b3406032d3a75269838c9d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.338506 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.338472 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53f1b9e34b3406032d3a75269838c9d7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal\" (UID: \"53f1b9e34b3406032d3a75269838c9d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.338506 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.338493 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a98b158e97d3e00d1e6f3fd6fce59404-config\") pod \"kube-apiserver-proxy-ip-10-0-131-148.ec2.internal\" (UID: \"a98b158e97d3e00d1e6f3fd6fce59404\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.338591 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.338569 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53f1b9e34b3406032d3a75269838c9d7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal\" (UID: \"53f1b9e34b3406032d3a75269838c9d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.338629 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.338591 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a98b158e97d3e00d1e6f3fd6fce59404-config\") pod \"kube-apiserver-proxy-ip-10-0-131-148.ec2.internal\" (UID: \"a98b158e97d3e00d1e6f3fd6fce59404\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.338629 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.338568 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/53f1b9e34b3406032d3a75269838c9d7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal\" (UID: \"53f1b9e34b3406032d3a75269838c9d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.437542 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:24.437476 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 31 22:57:24.541886 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:24.541815 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 31 22:57:24.565794 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.565765 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.569811 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.569786 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" Jul 31 22:57:24.642682 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:24.642628 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 31 22:57:24.743676 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:24.743643 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 31 22:57:24.844201 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:24.844105 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 31 22:57:24.875397 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.875357 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 31 22:57:24.875903 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.875554 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 31 22:57:24.899383 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.899348 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 22:57:24.945157 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:24.945122 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 31 22:57:24.947878 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.947857 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 22:57:24.961512 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.961461 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-30 22:52:24 +0000 UTC" deadline="2028-02-02 12:47:15.847706507 +0000 UTC" Jul 31 22:57:24.961661 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:24.961503 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13213h49m50.886206639s" Jul 31 22:57:25.036637 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.036590 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 31 22:57:25.052079 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.052057 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 31 22:57:25.058070 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.058039 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" Jul 31 22:57:25.068676 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.068649 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 31 22:57:25.200190 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:25.200140 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53f1b9e34b3406032d3a75269838c9d7.slice/crio-278b676de75d816282f5c7064e32b77037e2da6db2d9f850ae7905d75d2e42bb WatchSource:0}: Error finding container 278b676de75d816282f5c7064e32b77037e2da6db2d9f850ae7905d75d2e42bb: Status 404 returned error can't find the container with id 278b676de75d816282f5c7064e32b77037e2da6db2d9f850ae7905d75d2e42bb Jul 31 22:57:25.200448 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:25.200428 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda98b158e97d3e00d1e6f3fd6fce59404.slice/crio-b9c5289eb950082b50598f7a42dfdbbc491f5f1c7b7fdfa9c104bea7437bba89 WatchSource:0}: Error finding container b9c5289eb950082b50598f7a42dfdbbc491f5f1c7b7fdfa9c104bea7437bba89: Status 404 returned error can't find the container with id b9c5289eb950082b50598f7a42dfdbbc491f5f1c7b7fdfa9c104bea7437bba89 Jul 31 22:57:25.204981 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.204966 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 31 22:57:25.497039 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.496947 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 22:57:25.918859 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.918775 2561 apiserver.go:52] "Watching apiserver" Jul 31 22:57:25.927960 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.927632 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 31 22:57:25.928299 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.928127 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal","openshift-network-diagnostics/network-check-target-9g22p","openshift-network-operator/iptables-alerter-sjtn4","kube-system/konnectivity-agent-dqrvj","kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal","openshift-cluster-node-tuning-operator/tuned-wbqzc","openshift-dns/node-resolver-5t5xj","openshift-multus/multus-additional-cni-plugins-p6gvt","openshift-multus/multus-sv97r","openshift-multus/network-metrics-daemon-44zk2","openshift-ovn-kubernetes/ovnkube-node-tlbsb","kube-system/global-pull-secret-syncer-lqgnq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb","openshift-image-registry/node-ca-lzdpn"] Jul 31 22:57:25.931266 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.930855 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5t5xj" Jul 31 22:57:25.934511 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.933493 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.934511 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.934038 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 31 22:57:25.934511 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.934077 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qwzbq\"" Jul 31 22:57:25.934771 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.934548 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 31 22:57:25.937007 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.936247 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 31 22:57:25.939115 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.938799 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sjtn4" Jul 31 22:57:25.939227 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.939122 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 31 22:57:25.942051 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.939373 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 31 22:57:25.942051 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.939389 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-f5q7j\"" Jul 31 22:57:25.942051 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.939545 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 31 22:57:25.942051 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.939880 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 31 22:57:25.942051 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.940683 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 31 22:57:25.942051 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.940709 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 31 22:57:25.943121 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.943046 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dqrvj" Jul 31 22:57:25.944818 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.944796 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-57n9m\"" Jul 31 22:57:25.944926 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.944857 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 31 22:57:25.945249 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.945206 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 31 22:57:25.946859 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.946658 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-g8hxx\"" Jul 31 22:57:25.946859 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.946717 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 31 22:57:25.946859 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.946820 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 31 22:57:25.947126 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.947107 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lzdpn" Jul 31 22:57:25.947598 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.947339 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 31 22:57:25.949498 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.949483 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:25.950808 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.950763 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 31 22:57:25.951813 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951125 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 31 22:57:25.951813 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951343 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 31 22:57:25.951813 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951482 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xvtzz\"" Jul 31 22:57:25.951813 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951549 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23539586-9f8f-426a-b2a0-e9dbd8a06f85-host-slash\") pod \"iptables-alerter-sjtn4\" (UID: \"23539586-9f8f-426a-b2a0-e9dbd8a06f85\") " pod="openshift-network-operator/iptables-alerter-sjtn4" Jul 31 22:57:25.951813 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951604 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntk9j\" (UniqueName: \"kubernetes.io/projected/5753fb43-dec6-43b9-806e-3c593edb98dd-kube-api-access-ntk9j\") pod \"node-resolver-5t5xj\" (UID: \"5753fb43-dec6-43b9-806e-3c593edb98dd\") " pod="openshift-dns/node-resolver-5t5xj" Jul 31 22:57:25.951813 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951633 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-run-systemd\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.951813 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951659 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-run-openvswitch\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.951813 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951707 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/815265b8-7f60-4832-81ce-d3f516f7b7c5-ovn-node-metrics-cert\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.951813 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951741 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/23539586-9f8f-426a-b2a0-e9dbd8a06f85-iptables-alerter-script\") pod \"iptables-alerter-sjtn4\" (UID: \"23539586-9f8f-426a-b2a0-e9dbd8a06f85\") " pod="openshift-network-operator/iptables-alerter-sjtn4" Jul 31 22:57:25.951813 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951773 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5753fb43-dec6-43b9-806e-3c593edb98dd-hosts-file\") pod \"node-resolver-5t5xj\" (UID: \"5753fb43-dec6-43b9-806e-3c593edb98dd\") " pod="openshift-dns/node-resolver-5t5xj" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951846 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-run-netns\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951879 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-run-ovn\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951904 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-node-log\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951928 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-run-ovn-kubernetes\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951953 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2bd27fa7-9907-419d-af3b-43b77074de67-agent-certs\") pod \"konnectivity-agent-dqrvj\" (UID: \"2bd27fa7-9907-419d-af3b-43b77074de67\") " pod="kube-system/konnectivity-agent-dqrvj" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951987 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-var-lib-openvswitch\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.951994 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sv97r" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.952012 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzdbc\" (UniqueName: \"kubernetes.io/projected/23539586-9f8f-426a-b2a0-e9dbd8a06f85-kube-api-access-fzdbc\") pod \"iptables-alerter-sjtn4\" (UID: \"23539586-9f8f-426a-b2a0-e9dbd8a06f85\") " pod="openshift-network-operator/iptables-alerter-sjtn4" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.952037 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2bd27fa7-9907-419d-af3b-43b77074de67-konnectivity-ca\") pod \"konnectivity-agent-dqrvj\" (UID: \"2bd27fa7-9907-419d-af3b-43b77074de67\") " pod="kube-system/konnectivity-agent-dqrvj" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.952063 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-etc-openvswitch\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.952087 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-kubelet\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.952109 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg4kt\" (UniqueName: \"kubernetes.io/projected/815265b8-7f60-4832-81ce-d3f516f7b7c5-kube-api-access-fg4kt\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.952138 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-slash\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.952161 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-cni-netd\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.952186 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.952242 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/815265b8-7f60-4832-81ce-d3f516f7b7c5-ovnkube-script-lib\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.952359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.952268 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5753fb43-dec6-43b9-806e-3c593edb98dd-tmp-dir\") pod \"node-resolver-5t5xj\" (UID: \"5753fb43-dec6-43b9-806e-3c593edb98dd\") " pod="openshift-dns/node-resolver-5t5xj" Jul 31 22:57:25.953390 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.952296 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-systemd-units\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.953390 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.952321 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-log-socket\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.953390 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.952345 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/815265b8-7f60-4832-81ce-d3f516f7b7c5-ovnkube-config\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.953390 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.952391 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/815265b8-7f60-4832-81ce-d3f516f7b7c5-env-overrides\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.953390 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.952416 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-cni-bin\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:25.955973 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.954782 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 31 22:57:25.955973 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.954785 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:25.955973 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.955091 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-56w8s\"" Jul 31 22:57:25.955973 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.955322 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 31 22:57:25.955973 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.955522 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 31 22:57:25.955973 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.955659 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 31 22:57:25.955973 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.955671 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 31 22:57:25.955973 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.955899 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 31 22:57:25.956462 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.956009 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6g9hb\"" Jul 31 22:57:25.956462 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.956149 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 31 22:57:25.957540 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.957509 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:25.957649 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:25.957614 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:25.960286 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.960267 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:25.960412 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:25.960346 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:25.960522 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.960484 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 31 22:57:25.961749 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.961445 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 31 22:57:25.963371 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.963348 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2wmdp\"" Jul 31 22:57:25.963627 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.963608 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 31 22:57:25.963696 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.963646 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:25.965521 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.965497 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:25.965647 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:25.965593 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:25.966409 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.966385 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zr898\"" Jul 31 22:57:25.966759 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.966732 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 31 22:57:25.966954 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.966936 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 31 22:57:25.984566 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.984518 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tjg9t" Jul 31 22:57:25.990791 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:25.990756 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tjg9t" Jul 31 22:57:26.037639 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.037612 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 31 22:57:26.054492 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054453 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:26.054709 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054536 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c51d9ed-66ce-4160-8f08-bc20664c47b5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.054709 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054575 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-multus-conf-dir\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.054709 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054603 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-run-multus-certs\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.054709 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054636 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/815265b8-7f60-4832-81ce-d3f516f7b7c5-ovnkube-script-lib\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.054709 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054659 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d85e5451-3e8a-46bb-beb6-883353da3494-serviceca\") pod \"node-ca-lzdpn\" (UID: \"d85e5451-3e8a-46bb-beb6-883353da3494\") " pod="openshift-image-registry/node-ca-lzdpn" Jul 31 22:57:26.054709 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054675 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-multus-cni-dir\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.054709 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054703 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-var-lib-kubelet\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.055090 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054731 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-sys\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.055090 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054758 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-cni-bin\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.055090 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054785 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4c51d9ed-66ce-4160-8f08-bc20664c47b5-os-release\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.055090 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054808 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/40c30d12-8f96-4d63-a17e-5a3c101f2439-dbus\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:26.055090 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054833 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23539586-9f8f-426a-b2a0-e9dbd8a06f85-host-slash\") pod \"iptables-alerter-sjtn4\" (UID: \"23539586-9f8f-426a-b2a0-e9dbd8a06f85\") " pod="openshift-network-operator/iptables-alerter-sjtn4" Jul 31 22:57:26.055090 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054861 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntk9j\" (UniqueName: \"kubernetes.io/projected/5753fb43-dec6-43b9-806e-3c593edb98dd-kube-api-access-ntk9j\") pod \"node-resolver-5t5xj\" (UID: \"5753fb43-dec6-43b9-806e-3c593edb98dd\") " pod="openshift-dns/node-resolver-5t5xj" Jul 31 22:57:26.055090 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054909 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-run-systemd\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.055090 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054934 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-run-openvswitch\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.055090 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054959 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-registration-dir\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.055090 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054977 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-device-dir\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.055090 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.054994 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-modprobe-d\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.055090 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055018 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/23539586-9f8f-426a-b2a0-e9dbd8a06f85-iptables-alerter-script\") pod \"iptables-alerter-sjtn4\" (UID: \"23539586-9f8f-426a-b2a0-e9dbd8a06f85\") " pod="openshift-network-operator/iptables-alerter-sjtn4" Jul 31 22:57:26.055090 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055045 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-node-log\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.055090 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055072 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2bd27fa7-9907-419d-af3b-43b77074de67-agent-certs\") pod \"konnectivity-agent-dqrvj\" (UID: \"2bd27fa7-9907-419d-af3b-43b77074de67\") " pod="kube-system/konnectivity-agent-dqrvj" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055098 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2bd27fa7-9907-419d-af3b-43b77074de67-konnectivity-ca\") pod \"konnectivity-agent-dqrvj\" (UID: \"2bd27fa7-9907-419d-af3b-43b77074de67\") " pod="kube-system/konnectivity-agent-dqrvj" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055131 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-system-cni-dir\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055154 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-run\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055155 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23539586-9f8f-426a-b2a0-e9dbd8a06f85-host-slash\") pod \"iptables-alerter-sjtn4\" (UID: \"23539586-9f8f-426a-b2a0-e9dbd8a06f85\") " pod="openshift-network-operator/iptables-alerter-sjtn4" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055178 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-host\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055248 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e82a937d-4b34-4e32-8c73-76c56758aa66-tmp\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055283 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzdbc\" (UniqueName: \"kubernetes.io/projected/23539586-9f8f-426a-b2a0-e9dbd8a06f85-kube-api-access-fzdbc\") pod \"iptables-alerter-sjtn4\" (UID: \"23539586-9f8f-426a-b2a0-e9dbd8a06f85\") " pod="openshift-network-operator/iptables-alerter-sjtn4" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055314 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ggrl\" (UniqueName: \"kubernetes.io/projected/4c51d9ed-66ce-4160-8f08-bc20664c47b5-kube-api-access-7ggrl\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055343 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/40c30d12-8f96-4d63-a17e-5a3c101f2439-kubelet-config\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055372 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-etc-openvswitch\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055390 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/815265b8-7f60-4832-81ce-d3f516f7b7c5-ovnkube-script-lib\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055461 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-cni-bin\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055399 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-systemd\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055542 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-tuned\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055606 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-run-systemd\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055624 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcrcj\" (UniqueName: \"kubernetes.io/projected/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-kube-api-access-wcrcj\") pod \"network-metrics-daemon-44zk2\" (UID: \"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e\") " pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:26.055782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055608 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-etc-openvswitch\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055659 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-run-openvswitch\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055691 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg4kt\" (UniqueName: \"kubernetes.io/projected/815265b8-7f60-4832-81ce-d3f516f7b7c5-kube-api-access-fg4kt\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055724 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c51d9ed-66ce-4160-8f08-bc20664c47b5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055755 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-run-k8s-cni-cncf-io\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055779 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-var-lib-cni-bin\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055793 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/23539586-9f8f-426a-b2a0-e9dbd8a06f85-iptables-alerter-script\") pod \"iptables-alerter-sjtn4\" (UID: \"23539586-9f8f-426a-b2a0-e9dbd8a06f85\") " pod="openshift-network-operator/iptables-alerter-sjtn4" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055803 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/02a7725c-7910-4311-bba9-dacdbc0d7ce5-multus-daemon-config\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055843 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-sysctl-d\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055875 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-slash\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055881 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-node-log\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055895 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-cni-netd\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055922 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055969 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.055989 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-cni-netd\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056004 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-slash\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056019 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-cnibin\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.056680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056049 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-etc-kubernetes\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056074 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-socket-dir\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056101 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-etc-selinux\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056133 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-222nm\" (UniqueName: \"kubernetes.io/projected/e82a937d-4b34-4e32-8c73-76c56758aa66-kube-api-access-222nm\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056161 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5753fb43-dec6-43b9-806e-3c593edb98dd-tmp-dir\") pod \"node-resolver-5t5xj\" (UID: \"5753fb43-dec6-43b9-806e-3c593edb98dd\") " pod="openshift-dns/node-resolver-5t5xj" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056188 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-systemd-units\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056210 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2bd27fa7-9907-419d-af3b-43b77074de67-konnectivity-ca\") pod \"konnectivity-agent-dqrvj\" (UID: \"2bd27fa7-9907-419d-af3b-43b77074de67\") " pod="kube-system/konnectivity-agent-dqrvj" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056213 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-log-socket\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056265 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/815265b8-7f60-4832-81ce-d3f516f7b7c5-ovnkube-config\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056272 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-systemd-units\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056290 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/815265b8-7f60-4832-81ce-d3f516f7b7c5-env-overrides\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056315 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-log-socket\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056318 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c51d9ed-66ce-4160-8f08-bc20664c47b5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056349 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-hostroot\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056372 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-sys-fs\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056396 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c51d9ed-66ce-4160-8f08-bc20664c47b5-cni-binary-copy\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056425 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 31 22:57:26.057612 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056461 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-run-netns\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056490 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-var-lib-cni-multus\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056518 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9dlq\" (UniqueName: \"kubernetes.io/projected/02a7725c-7910-4311-bba9-dacdbc0d7ce5-kube-api-access-b9dlq\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056548 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/815265b8-7f60-4832-81ce-d3f516f7b7c5-ovn-node-metrics-cert\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056590 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c51d9ed-66ce-4160-8f08-bc20664c47b5-cnibin\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056614 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d85e5451-3e8a-46bb-beb6-883353da3494-host\") pod \"node-ca-lzdpn\" (UID: \"d85e5451-3e8a-46bb-beb6-883353da3494\") " pod="openshift-image-registry/node-ca-lzdpn" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056639 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056665 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zbrk\" (UniqueName: \"kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk\") pod \"network-check-target-9g22p\" (UID: \"cd1248a5-5ddf-4647-b417-5099076f0133\") " pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056689 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-sysctl-conf\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056714 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs\") pod \"network-metrics-daemon-44zk2\" (UID: \"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e\") " pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056720 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/815265b8-7f60-4832-81ce-d3f516f7b7c5-env-overrides\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056742 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5753fb43-dec6-43b9-806e-3c593edb98dd-hosts-file\") pod \"node-resolver-5t5xj\" (UID: \"5753fb43-dec6-43b9-806e-3c593edb98dd\") " pod="openshift-dns/node-resolver-5t5xj" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056767 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-run-netns\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056792 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-run-ovn\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056820 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-run-ovn-kubernetes\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056841 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-var-lib-openvswitch\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056482 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5753fb43-dec6-43b9-806e-3c593edb98dd-tmp-dir\") pod \"node-resolver-5t5xj\" (UID: \"5753fb43-dec6-43b9-806e-3c593edb98dd\") " pod="openshift-dns/node-resolver-5t5xj" Jul 31 22:57:26.058357 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056876 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c51d9ed-66ce-4160-8f08-bc20664c47b5-system-cni-dir\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056909 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5753fb43-dec6-43b9-806e-3c593edb98dd-hosts-file\") pod \"node-resolver-5t5xj\" (UID: \"5753fb43-dec6-43b9-806e-3c593edb98dd\") " pod="openshift-dns/node-resolver-5t5xj" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056909 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqr2h\" (UniqueName: \"kubernetes.io/projected/6e72564a-f37c-4507-893d-093ec9898e6b-kube-api-access-zqr2h\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056946 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-sysconfig\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056972 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-kubernetes\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056996 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/815265b8-7f60-4832-81ce-d3f516f7b7c5-ovnkube-config\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.056997 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-var-lib-kubelet\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.057050 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-run-netns\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.057063 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-os-release\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.057092 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-multus-socket-dir-parent\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.057100 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-run-ovn\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.057117 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-lib-modules\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.057145 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-run-ovn-kubernetes\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.057192 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-var-lib-openvswitch\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.057147 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-kubelet\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.057228 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/815265b8-7f60-4832-81ce-d3f516f7b7c5-host-kubelet\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.057231 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msj96\" (UniqueName: \"kubernetes.io/projected/d85e5451-3e8a-46bb-beb6-883353da3494-kube-api-access-msj96\") pod \"node-ca-lzdpn\" (UID: \"d85e5451-3e8a-46bb-beb6-883353da3494\") " pod="openshift-image-registry/node-ca-lzdpn" Jul 31 22:57:26.059251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.057265 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02a7725c-7910-4311-bba9-dacdbc0d7ce5-cni-binary-copy\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.061817 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.061791 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2bd27fa7-9907-419d-af3b-43b77074de67-agent-certs\") pod \"konnectivity-agent-dqrvj\" (UID: \"2bd27fa7-9907-419d-af3b-43b77074de67\") " pod="kube-system/konnectivity-agent-dqrvj" Jul 31 22:57:26.062387 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.062365 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/815265b8-7f60-4832-81ce-d3f516f7b7c5-ovn-node-metrics-cert\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.066202 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.066093 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg4kt\" (UniqueName: \"kubernetes.io/projected/815265b8-7f60-4832-81ce-d3f516f7b7c5-kube-api-access-fg4kt\") pod \"ovnkube-node-tlbsb\" (UID: \"815265b8-7f60-4832-81ce-d3f516f7b7c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.066326 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.066224 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzdbc\" (UniqueName: \"kubernetes.io/projected/23539586-9f8f-426a-b2a0-e9dbd8a06f85-kube-api-access-fzdbc\") pod \"iptables-alerter-sjtn4\" (UID: \"23539586-9f8f-426a-b2a0-e9dbd8a06f85\") " pod="openshift-network-operator/iptables-alerter-sjtn4" Jul 31 22:57:26.067387 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.067338 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntk9j\" (UniqueName: \"kubernetes.io/projected/5753fb43-dec6-43b9-806e-3c593edb98dd-kube-api-access-ntk9j\") pod \"node-resolver-5t5xj\" (UID: \"5753fb43-dec6-43b9-806e-3c593edb98dd\") " pod="openshift-dns/node-resolver-5t5xj" Jul 31 22:57:26.120842 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.120801 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 22:57:26.132002 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.131933 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" event={"ID":"53f1b9e34b3406032d3a75269838c9d7","Type":"ContainerStarted","Data":"278b676de75d816282f5c7064e32b77037e2da6db2d9f850ae7905d75d2e42bb"} Jul 31 22:57:26.133528 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.133489 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" event={"ID":"a98b158e97d3e00d1e6f3fd6fce59404","Type":"ContainerStarted","Data":"b9c5289eb950082b50598f7a42dfdbbc491f5f1c7b7fdfa9c104bea7437bba89"} Jul 31 22:57:26.158622 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158579 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c51d9ed-66ce-4160-8f08-bc20664c47b5-system-cni-dir\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.158622 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158625 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqr2h\" (UniqueName: \"kubernetes.io/projected/6e72564a-f37c-4507-893d-093ec9898e6b-kube-api-access-zqr2h\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.158892 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158655 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-sysconfig\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.158892 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158682 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-kubernetes\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.158892 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158704 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c51d9ed-66ce-4160-8f08-bc20664c47b5-system-cni-dir\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.158892 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158735 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-var-lib-kubelet\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.158892 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158760 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-os-release\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.158892 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158793 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-kubernetes\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.158892 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158793 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-sysconfig\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.158892 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158802 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-multus-socket-dir-parent\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.158892 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158824 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-var-lib-kubelet\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.158892 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158862 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-os-release\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.158892 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158865 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-lib-modules\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158906 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msj96\" (UniqueName: \"kubernetes.io/projected/d85e5451-3e8a-46bb-beb6-883353da3494-kube-api-access-msj96\") pod \"node-ca-lzdpn\" (UID: \"d85e5451-3e8a-46bb-beb6-883353da3494\") " pod="openshift-image-registry/node-ca-lzdpn" Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158929 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02a7725c-7910-4311-bba9-dacdbc0d7ce5-cni-binary-copy\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158953 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-multus-socket-dir-parent\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158955 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158969 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-lib-modules\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.158993 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c51d9ed-66ce-4160-8f08-bc20664c47b5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159021 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-multus-conf-dir\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159042 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-run-multus-certs\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159058 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d85e5451-3e8a-46bb-beb6-883353da3494-serviceca\") pod \"node-ca-lzdpn\" (UID: \"d85e5451-3e8a-46bb-beb6-883353da3494\") " pod="openshift-image-registry/node-ca-lzdpn" Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.159056 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159084 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-multus-cni-dir\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159105 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-multus-conf-dir\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159109 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-var-lib-kubelet\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.159153 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret podName:40c30d12-8f96-4d63-a17e-5a3c101f2439 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:26.659108907 +0000 UTC m=+3.131553074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret") pod "global-pull-secret-syncer-lqgnq" (UID: "40c30d12-8f96-4d63-a17e-5a3c101f2439") : object "kube-system"/"original-pull-secret" not registered Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159160 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-var-lib-kubelet\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159206 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-run-multus-certs\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.159515 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159241 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-sys\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159277 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4c51d9ed-66ce-4160-8f08-bc20664c47b5-os-release\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159304 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/40c30d12-8f96-4d63-a17e-5a3c101f2439-dbus\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159307 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c51d9ed-66ce-4160-8f08-bc20664c47b5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159339 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-registration-dir\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159369 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-device-dir\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159382 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-sys\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159439 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-device-dir\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159455 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4c51d9ed-66ce-4160-8f08-bc20664c47b5-os-release\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159471 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-modprobe-d\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159501 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-multus-cni-dir\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159513 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/40c30d12-8f96-4d63-a17e-5a3c101f2439-dbus\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159508 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-system-cni-dir\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159551 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-run\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159582 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-registration-dir\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159597 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-host\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159618 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-modprobe-d\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159629 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e82a937d-4b34-4e32-8c73-76c56758aa66-tmp\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.160362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159647 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d85e5451-3e8a-46bb-beb6-883353da3494-serviceca\") pod \"node-ca-lzdpn\" (UID: \"d85e5451-3e8a-46bb-beb6-883353da3494\") " pod="openshift-image-registry/node-ca-lzdpn" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159662 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ggrl\" (UniqueName: \"kubernetes.io/projected/4c51d9ed-66ce-4160-8f08-bc20664c47b5-kube-api-access-7ggrl\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159666 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02a7725c-7910-4311-bba9-dacdbc0d7ce5-cni-binary-copy\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159672 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-host\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159631 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-run\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159677 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-system-cni-dir\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159688 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/40c30d12-8f96-4d63-a17e-5a3c101f2439-kubelet-config\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159755 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-systemd\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159827 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-systemd\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159881 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/40c30d12-8f96-4d63-a17e-5a3c101f2439-kubelet-config\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159874 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-tuned\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159921 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcrcj\" (UniqueName: \"kubernetes.io/projected/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-kube-api-access-wcrcj\") pod \"network-metrics-daemon-44zk2\" (UID: \"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e\") " pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159955 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c51d9ed-66ce-4160-8f08-bc20664c47b5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.159984 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-run-k8s-cni-cncf-io\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160008 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-var-lib-cni-bin\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160033 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/02a7725c-7910-4311-bba9-dacdbc0d7ce5-multus-daemon-config\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160057 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-sysctl-d\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160080 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-run-k8s-cni-cncf-io\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.161261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160087 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-cnibin\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160121 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-etc-kubernetes\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160137 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-cnibin\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160151 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-socket-dir\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160179 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-etc-selinux\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160182 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-var-lib-cni-bin\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160332 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-etc-selinux\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160443 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-sysctl-d\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160479 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-222nm\" (UniqueName: \"kubernetes.io/projected/e82a937d-4b34-4e32-8c73-76c56758aa66-kube-api-access-222nm\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160514 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c51d9ed-66ce-4160-8f08-bc20664c47b5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160520 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c51d9ed-66ce-4160-8f08-bc20664c47b5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160542 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-hostroot\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160587 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-sys-fs\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160613 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c51d9ed-66ce-4160-8f08-bc20664c47b5-cni-binary-copy\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160638 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-run-netns\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160663 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-var-lib-cni-multus\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160687 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9dlq\" (UniqueName: \"kubernetes.io/projected/02a7725c-7910-4311-bba9-dacdbc0d7ce5-kube-api-access-b9dlq\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.162261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160698 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/02a7725c-7910-4311-bba9-dacdbc0d7ce5-multus-daemon-config\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160712 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c51d9ed-66ce-4160-8f08-bc20664c47b5-cnibin\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160751 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d85e5451-3e8a-46bb-beb6-883353da3494-host\") pod \"node-ca-lzdpn\" (UID: \"d85e5451-3e8a-46bb-beb6-883353da3494\") " pod="openshift-image-registry/node-ca-lzdpn" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160778 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160793 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-run-netns\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160811 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zbrk\" (UniqueName: \"kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk\") pod \"network-check-target-9g22p\" (UID: \"cd1248a5-5ddf-4647-b417-5099076f0133\") " pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160833 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d85e5451-3e8a-46bb-beb6-883353da3494-host\") pod \"node-ca-lzdpn\" (UID: \"d85e5451-3e8a-46bb-beb6-883353da3494\") " pod="openshift-image-registry/node-ca-lzdpn" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160871 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-sysctl-conf\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.160900 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs\") pod \"network-metrics-daemon-44zk2\" (UID: \"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e\") " pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.161170 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.161207 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c51d9ed-66ce-4160-8f08-bc20664c47b5-cni-binary-copy\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.161215 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c51d9ed-66ce-4160-8f08-bc20664c47b5-cnibin\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.161271 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-host-var-lib-cni-multus\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.161269 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-hostroot\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.161311 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02a7725c-7910-4311-bba9-dacdbc0d7ce5-etc-kubernetes\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.161316 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c51d9ed-66ce-4160-8f08-bc20664c47b5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.161327 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-sys-fs\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.163139 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.161405 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6e72564a-f37c-4507-893d-093ec9898e6b-socket-dir\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.163913 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.161452 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-sysctl-conf\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.163913 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.161588 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:57:26.163913 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.161649 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs podName:58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e nodeName:}" failed. No retries permitted until 2026-07-31 22:57:26.661631644 +0000 UTC m=+3.134075812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs") pod "network-metrics-daemon-44zk2" (UID: "58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:57:26.164057 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.164030 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e82a937d-4b34-4e32-8c73-76c56758aa66-tmp\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.164645 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.164617 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e82a937d-4b34-4e32-8c73-76c56758aa66-etc-tuned\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.179831 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.179535 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 22:57:26.179831 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.179569 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 22:57:26.179831 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.179584 2561 projected.go:196] Error preparing data for projected volume kube-api-access-2zbrk for pod openshift-network-diagnostics/network-check-target-9g22p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 22:57:26.179831 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.179653 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk podName:cd1248a5-5ddf-4647-b417-5099076f0133 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:26.679632258 +0000 UTC m=+3.152076424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2zbrk" (UniqueName: "kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk") pod "network-check-target-9g22p" (UID: "cd1248a5-5ddf-4647-b417-5099076f0133") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 22:57:26.180684 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.180656 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcrcj\" (UniqueName: \"kubernetes.io/projected/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-kube-api-access-wcrcj\") pod \"network-metrics-daemon-44zk2\" (UID: \"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e\") " pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:26.180787 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.180694 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9dlq\" (UniqueName: \"kubernetes.io/projected/02a7725c-7910-4311-bba9-dacdbc0d7ce5-kube-api-access-b9dlq\") pod \"multus-sv97r\" (UID: \"02a7725c-7910-4311-bba9-dacdbc0d7ce5\") " pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.183253 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.183205 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msj96\" (UniqueName: \"kubernetes.io/projected/d85e5451-3e8a-46bb-beb6-883353da3494-kube-api-access-msj96\") pod \"node-ca-lzdpn\" (UID: \"d85e5451-3e8a-46bb-beb6-883353da3494\") " pod="openshift-image-registry/node-ca-lzdpn" Jul 31 22:57:26.194786 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.194721 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ggrl\" (UniqueName: \"kubernetes.io/projected/4c51d9ed-66ce-4160-8f08-bc20664c47b5-kube-api-access-7ggrl\") pod \"multus-additional-cni-plugins-p6gvt\" (UID: \"4c51d9ed-66ce-4160-8f08-bc20664c47b5\") " pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.195394 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.195345 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-222nm\" (UniqueName: \"kubernetes.io/projected/e82a937d-4b34-4e32-8c73-76c56758aa66-kube-api-access-222nm\") pod \"tuned-wbqzc\" (UID: \"e82a937d-4b34-4e32-8c73-76c56758aa66\") " pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.195797 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.195774 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqr2h\" (UniqueName: \"kubernetes.io/projected/6e72564a-f37c-4507-893d-093ec9898e6b-kube-api-access-zqr2h\") pod \"aws-ebs-csi-driver-node-qtfcb\" (UID: \"6e72564a-f37c-4507-893d-093ec9898e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.249328 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.249288 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5t5xj" Jul 31 22:57:26.257536 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.257433 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:26.271520 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.271488 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sjtn4" Jul 31 22:57:26.278540 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.278500 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dqrvj" Jul 31 22:57:26.289621 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.289600 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lzdpn" Jul 31 22:57:26.301642 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.301618 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" Jul 31 22:57:26.312598 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.312574 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sv97r" Jul 31 22:57:26.328666 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.328637 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" Jul 31 22:57:26.336803 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.336778 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p6gvt" Jul 31 22:57:26.665130 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.665045 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs\") pod \"network-metrics-daemon-44zk2\" (UID: \"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e\") " pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:26.665130 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.665100 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:26.665376 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.665241 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 22:57:26.665376 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.665330 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret podName:40c30d12-8f96-4d63-a17e-5a3c101f2439 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:27.665291541 +0000 UTC m=+4.137735719 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret") pod "global-pull-secret-syncer-lqgnq" (UID: "40c30d12-8f96-4d63-a17e-5a3c101f2439") : object "kube-system"/"original-pull-secret" not registered Jul 31 22:57:26.665376 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.665242 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:57:26.665574 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.665434 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs podName:58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e nodeName:}" failed. No retries permitted until 2026-07-31 22:57:27.665400661 +0000 UTC m=+4.137844824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs") pod "network-metrics-daemon-44zk2" (UID: "58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:57:26.766366 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.766320 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zbrk\" (UniqueName: \"kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk\") pod \"network-check-target-9g22p\" (UID: \"cd1248a5-5ddf-4647-b417-5099076f0133\") " pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:26.766575 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.766525 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 22:57:26.766575 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.766548 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 22:57:26.766575 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.766560 2561 projected.go:196] Error preparing data for projected volume kube-api-access-2zbrk for pod openshift-network-diagnostics/network-check-target-9g22p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 22:57:26.766722 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:26.766636 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk podName:cd1248a5-5ddf-4647-b417-5099076f0133 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:27.76661967 +0000 UTC m=+4.239063832 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2zbrk" (UniqueName: "kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk") pod "network-check-target-9g22p" (UID: "cd1248a5-5ddf-4647-b417-5099076f0133") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 22:57:26.994265 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.994135 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 22:52:26 +0000 UTC" deadline="2028-01-14 19:50:37.030489385 +0000 UTC" Jul 31 22:57:26.994265 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:26.994199 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12764h53m10.036294371s" Jul 31 22:57:27.126444 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:27.126406 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:27.126619 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:27.126538 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:27.126619 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:27.126577 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:27.126786 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:27.126631 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:27.360320 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:27.360286 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode82a937d_4b34_4e32_8c73_76c56758aa66.slice/crio-79ce32c04b2f557d60dc1cee05af66a2688ec802e8d1be6b07774de38aec0fed WatchSource:0}: Error finding container 79ce32c04b2f557d60dc1cee05af66a2688ec802e8d1be6b07774de38aec0fed: Status 404 returned error can't find the container with id 79ce32c04b2f557d60dc1cee05af66a2688ec802e8d1be6b07774de38aec0fed Jul 31 22:57:27.362951 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:27.362919 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod815265b8_7f60_4832_81ce_d3f516f7b7c5.slice/crio-031d094b04ca3106a51eeaa78c96bdb9dd2100a1d8c19f5b9457a7b94d372baa WatchSource:0}: Error finding container 031d094b04ca3106a51eeaa78c96bdb9dd2100a1d8c19f5b9457a7b94d372baa: Status 404 returned error can't find the container with id 031d094b04ca3106a51eeaa78c96bdb9dd2100a1d8c19f5b9457a7b94d372baa Jul 31 22:57:27.366962 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:27.366899 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23539586_9f8f_426a_b2a0_e9dbd8a06f85.slice/crio-25eb2a1516b5fcf1e6b82becd0dacd981cda6c83c8608f936ca7729b4b8bf606 WatchSource:0}: Error finding container 25eb2a1516b5fcf1e6b82becd0dacd981cda6c83c8608f936ca7729b4b8bf606: Status 404 returned error can't find the container with id 25eb2a1516b5fcf1e6b82becd0dacd981cda6c83c8608f936ca7729b4b8bf606 Jul 31 22:57:27.367783 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:27.367765 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd85e5451_3e8a_46bb_beb6_883353da3494.slice/crio-28794cbaab76b8e188a77a6c8abae6a9532388971c562090c267cb6755e43c3b WatchSource:0}: Error finding container 28794cbaab76b8e188a77a6c8abae6a9532388971c562090c267cb6755e43c3b: Status 404 returned error can't find the container with id 28794cbaab76b8e188a77a6c8abae6a9532388971c562090c267cb6755e43c3b Jul 31 22:57:27.369158 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:27.369131 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c51d9ed_66ce_4160_8f08_bc20664c47b5.slice/crio-1b33093a5c0453912e95a93512f62b5f65ceaaac25d4506de1c1b0712fae5218 WatchSource:0}: Error finding container 1b33093a5c0453912e95a93512f62b5f65ceaaac25d4506de1c1b0712fae5218: Status 404 returned error can't find the container with id 1b33093a5c0453912e95a93512f62b5f65ceaaac25d4506de1c1b0712fae5218 Jul 31 22:57:27.369620 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:27.369596 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5753fb43_dec6_43b9_806e_3c593edb98dd.slice/crio-0418d0a458ff1bccd09a1ced4a848482726f9ec582fce39ebe81e3e52c20872a WatchSource:0}: Error finding container 0418d0a458ff1bccd09a1ced4a848482726f9ec582fce39ebe81e3e52c20872a: Status 404 returned error can't find the container with id 0418d0a458ff1bccd09a1ced4a848482726f9ec582fce39ebe81e3e52c20872a Jul 31 22:57:27.371084 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:27.371030 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02a7725c_7910_4311_bba9_dacdbc0d7ce5.slice/crio-f6fcba78707c45d1bfc05aec1204502fe3d3e934096718fc94d9ee81d0b78ef7 WatchSource:0}: Error finding container f6fcba78707c45d1bfc05aec1204502fe3d3e934096718fc94d9ee81d0b78ef7: Status 404 returned error can't find the container with id f6fcba78707c45d1bfc05aec1204502fe3d3e934096718fc94d9ee81d0b78ef7 Jul 31 22:57:27.373365 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:57:27.372221 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd27fa7_9907_419d_af3b_43b77074de67.slice/crio-c02eec6c5ec8c20bd33fc4af65b49e25a640d0fd916c8ca1e812d468f4f9229c WatchSource:0}: Error finding container c02eec6c5ec8c20bd33fc4af65b49e25a640d0fd916c8ca1e812d468f4f9229c: Status 404 returned error can't find the container with id c02eec6c5ec8c20bd33fc4af65b49e25a640d0fd916c8ca1e812d468f4f9229c Jul 31 22:57:27.690061 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:27.690015 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs\") pod \"network-metrics-daemon-44zk2\" (UID: \"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e\") " pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:27.690252 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:27.690073 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:27.690252 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:27.690175 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:57:27.690252 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:27.690201 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 22:57:27.690252 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:27.690252 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs podName:58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e nodeName:}" failed. No retries permitted until 2026-07-31 22:57:29.690232328 +0000 UTC m=+6.162676496 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs") pod "network-metrics-daemon-44zk2" (UID: "58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:57:27.690411 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:27.690267 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret podName:40c30d12-8f96-4d63-a17e-5a3c101f2439 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:29.690260401 +0000 UTC m=+6.162704562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret") pod "global-pull-secret-syncer-lqgnq" (UID: "40c30d12-8f96-4d63-a17e-5a3c101f2439") : object "kube-system"/"original-pull-secret" not registered Jul 31 22:57:27.791054 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:27.791010 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zbrk\" (UniqueName: \"kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk\") pod \"network-check-target-9g22p\" (UID: \"cd1248a5-5ddf-4647-b417-5099076f0133\") " pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:27.791264 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:27.791205 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 22:57:27.791264 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:27.791236 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 22:57:27.791264 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:27.791253 2561 projected.go:196] Error preparing data for projected volume kube-api-access-2zbrk for pod openshift-network-diagnostics/network-check-target-9g22p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 22:57:27.791429 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:27.791330 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk podName:cd1248a5-5ddf-4647-b417-5099076f0133 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:29.791309676 +0000 UTC m=+6.263753857 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2zbrk" (UniqueName: "kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk") pod "network-check-target-9g22p" (UID: "cd1248a5-5ddf-4647-b417-5099076f0133") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 22:57:27.994478 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:27.994431 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 22:52:26 +0000 UTC" deadline="2028-05-07 21:48:00.799752984 +0000 UTC" Jul 31 22:57:27.994478 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:27.994476 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15502h50m32.805282406s" Jul 31 22:57:28.129023 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:28.128991 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:28.129206 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:28.129125 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:28.147184 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:28.147152 2561 generic.go:356] "Generic (PLEG): container finished" podID="53f1b9e34b3406032d3a75269838c9d7" containerID="dc8c44b993e6c9342c45860e4beafae3ac3014b280b26dabfcd8479420b8a043" exitCode=0 Jul 31 22:57:28.147368 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:28.147229 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" event={"ID":"53f1b9e34b3406032d3a75269838c9d7","Type":"ContainerDied","Data":"dc8c44b993e6c9342c45860e4beafae3ac3014b280b26dabfcd8479420b8a043"} Jul 31 22:57:28.170343 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:28.170305 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" event={"ID":"a98b158e97d3e00d1e6f3fd6fce59404","Type":"ContainerStarted","Data":"85441e0a6fa7fed4c2c43e4cd77b8a902dce612d5e6a66e4fedadba53c62b163"} Jul 31 22:57:28.179249 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:28.179215 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" event={"ID":"6e72564a-f37c-4507-893d-093ec9898e6b","Type":"ContainerStarted","Data":"05ceb04a8e97e68839d4a753110669df2f37941988d4437085978eac8fe76ecd"} Jul 31 22:57:28.195538 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:28.194740 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" podStartSLOduration=3.194716567 podStartE2EDuration="3.194716567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:57:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:28.193719219 +0000 UTC m=+4.666163405" watchObservedRunningTime="2026-07-31 22:57:28.194716567 +0000 UTC m=+4.667160753" Jul 31 22:57:28.197405 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:28.197309 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sv97r" event={"ID":"02a7725c-7910-4311-bba9-dacdbc0d7ce5","Type":"ContainerStarted","Data":"f6fcba78707c45d1bfc05aec1204502fe3d3e934096718fc94d9ee81d0b78ef7"} Jul 31 22:57:28.212432 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:28.212396 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sjtn4" event={"ID":"23539586-9f8f-426a-b2a0-e9dbd8a06f85","Type":"ContainerStarted","Data":"25eb2a1516b5fcf1e6b82becd0dacd981cda6c83c8608f936ca7729b4b8bf606"} Jul 31 22:57:28.229761 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:28.229723 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" event={"ID":"815265b8-7f60-4832-81ce-d3f516f7b7c5","Type":"ContainerStarted","Data":"031d094b04ca3106a51eeaa78c96bdb9dd2100a1d8c19f5b9457a7b94d372baa"} Jul 31 22:57:28.239789 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:28.239753 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dqrvj" event={"ID":"2bd27fa7-9907-419d-af3b-43b77074de67","Type":"ContainerStarted","Data":"c02eec6c5ec8c20bd33fc4af65b49e25a640d0fd916c8ca1e812d468f4f9229c"} Jul 31 22:57:28.249461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:28.249423 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5t5xj" event={"ID":"5753fb43-dec6-43b9-806e-3c593edb98dd","Type":"ContainerStarted","Data":"0418d0a458ff1bccd09a1ced4a848482726f9ec582fce39ebe81e3e52c20872a"} Jul 31 22:57:28.254959 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:28.254867 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6gvt" event={"ID":"4c51d9ed-66ce-4160-8f08-bc20664c47b5","Type":"ContainerStarted","Data":"1b33093a5c0453912e95a93512f62b5f65ceaaac25d4506de1c1b0712fae5218"} Jul 31 22:57:28.259861 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:28.259828 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lzdpn" event={"ID":"d85e5451-3e8a-46bb-beb6-883353da3494","Type":"ContainerStarted","Data":"28794cbaab76b8e188a77a6c8abae6a9532388971c562090c267cb6755e43c3b"} Jul 31 22:57:28.272609 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:28.272571 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" event={"ID":"e82a937d-4b34-4e32-8c73-76c56758aa66","Type":"ContainerStarted","Data":"79ce32c04b2f557d60dc1cee05af66a2688ec802e8d1be6b07774de38aec0fed"} Jul 31 22:57:29.127802 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:29.127000 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:29.127802 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:29.127132 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:29.127802 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:29.127587 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:29.127802 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:29.127696 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:29.300473 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:29.300011 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" event={"ID":"53f1b9e34b3406032d3a75269838c9d7","Type":"ContainerStarted","Data":"dfb573b1ddb00d1d49fdcabe450fa0ab540d201795db98d003c8c2f2dfddec2e"} Jul 31 22:57:29.318085 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:29.318029 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" podStartSLOduration=4.3180088340000005 podStartE2EDuration="4.318008834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:57:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:29.317769201 +0000 UTC m=+5.790213385" watchObservedRunningTime="2026-07-31 22:57:29.318008834 +0000 UTC m=+5.790453019" Jul 31 22:57:29.708314 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:29.708272 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs\") pod \"network-metrics-daemon-44zk2\" (UID: \"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e\") " pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:29.708577 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:29.708335 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:29.708577 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:29.708460 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 22:57:29.708577 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:29.708539 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret podName:40c30d12-8f96-4d63-a17e-5a3c101f2439 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:33.708519873 +0000 UTC m=+10.180964039 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret") pod "global-pull-secret-syncer-lqgnq" (UID: "40c30d12-8f96-4d63-a17e-5a3c101f2439") : object "kube-system"/"original-pull-secret" not registered Jul 31 22:57:29.708949 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:29.708931 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:57:29.709026 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:29.708984 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs podName:58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e nodeName:}" failed. No retries permitted until 2026-07-31 22:57:33.708967863 +0000 UTC m=+10.181412031 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs") pod "network-metrics-daemon-44zk2" (UID: "58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:57:29.809427 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:29.809385 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zbrk\" (UniqueName: \"kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk\") pod \"network-check-target-9g22p\" (UID: \"cd1248a5-5ddf-4647-b417-5099076f0133\") " pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:29.809639 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:29.809621 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 22:57:29.809720 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:29.809647 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 22:57:29.809720 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:29.809660 2561 projected.go:196] Error preparing data for projected volume kube-api-access-2zbrk for pod openshift-network-diagnostics/network-check-target-9g22p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 22:57:29.809824 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:29.809724 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk podName:cd1248a5-5ddf-4647-b417-5099076f0133 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:33.809705704 +0000 UTC m=+10.282149868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2zbrk" (UniqueName: "kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk") pod "network-check-target-9g22p" (UID: "cd1248a5-5ddf-4647-b417-5099076f0133") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 22:57:30.129820 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:30.129701 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:30.130280 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:30.129828 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:31.127136 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:31.126383 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:31.127136 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:31.126555 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:31.127136 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:31.126982 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:31.127136 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:31.127083 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:32.128898 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:32.128269 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:32.128898 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:32.128405 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:33.126849 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:33.126782 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:33.126849 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:33.126801 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:33.127104 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:33.126931 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:33.127388 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:33.127326 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:33.744431 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:33.743815 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs\") pod \"network-metrics-daemon-44zk2\" (UID: \"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e\") " pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:33.744431 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:33.743878 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:33.744431 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:33.744010 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 22:57:33.744431 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:33.744081 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret podName:40c30d12-8f96-4d63-a17e-5a3c101f2439 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:41.744061879 +0000 UTC m=+18.216506055 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret") pod "global-pull-secret-syncer-lqgnq" (UID: "40c30d12-8f96-4d63-a17e-5a3c101f2439") : object "kube-system"/"original-pull-secret" not registered Jul 31 22:57:33.744431 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:33.744011 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:57:33.744431 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:33.744185 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs podName:58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e nodeName:}" failed. No retries permitted until 2026-07-31 22:57:41.744165047 +0000 UTC m=+18.216609225 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs") pod "network-metrics-daemon-44zk2" (UID: "58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:57:33.845296 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:33.845254 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zbrk\" (UniqueName: \"kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk\") pod \"network-check-target-9g22p\" (UID: \"cd1248a5-5ddf-4647-b417-5099076f0133\") " pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:33.846296 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:33.845983 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 22:57:33.846296 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:33.846020 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 22:57:33.846296 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:33.846051 2561 projected.go:196] Error preparing data for projected volume kube-api-access-2zbrk for pod openshift-network-diagnostics/network-check-target-9g22p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 22:57:33.846296 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:33.846130 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk podName:cd1248a5-5ddf-4647-b417-5099076f0133 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:41.846100659 +0000 UTC m=+18.318544832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2zbrk" (UniqueName: "kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk") pod "network-check-target-9g22p" (UID: "cd1248a5-5ddf-4647-b417-5099076f0133") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 22:57:34.127943 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:34.127398 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:34.127943 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:34.127538 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:35.127028 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:35.126493 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:35.127028 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:35.126523 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:35.127028 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:35.126664 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:35.127028 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:35.126725 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:36.126337 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:36.126299 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:36.126543 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:36.126412 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:37.126150 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:37.126117 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:37.126627 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:37.126263 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:37.126627 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:37.126328 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:37.126627 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:37.126450 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:38.127251 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:38.127217 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:38.127652 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:38.127336 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:39.127135 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:39.127096 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:39.127324 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:39.127230 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:39.127324 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:39.127287 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:39.127803 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:39.127407 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:40.127054 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:40.127017 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:40.127260 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:40.127152 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:41.126993 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:41.126954 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:41.127592 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:41.126997 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:41.127592 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:41.127100 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:41.127592 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:41.127199 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:41.811137 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:41.811088 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs\") pod \"network-metrics-daemon-44zk2\" (UID: \"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e\") " pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:41.811137 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:41.811138 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:41.811351 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:41.811240 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 22:57:41.811351 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:41.811287 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret podName:40c30d12-8f96-4d63-a17e-5a3c101f2439 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:57.811273726 +0000 UTC m=+34.283717888 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret") pod "global-pull-secret-syncer-lqgnq" (UID: "40c30d12-8f96-4d63-a17e-5a3c101f2439") : object "kube-system"/"original-pull-secret" not registered Jul 31 22:57:41.811351 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:41.811240 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:57:41.811489 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:41.811394 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs podName:58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e nodeName:}" failed. No retries permitted until 2026-07-31 22:57:57.811376737 +0000 UTC m=+34.283820914 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs") pod "network-metrics-daemon-44zk2" (UID: "58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:57:41.911766 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:41.911729 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zbrk\" (UniqueName: \"kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk\") pod \"network-check-target-9g22p\" (UID: \"cd1248a5-5ddf-4647-b417-5099076f0133\") " pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:41.911937 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:41.911914 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 22:57:41.912002 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:41.911941 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 22:57:41.912002 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:41.911953 2561 projected.go:196] Error preparing data for projected volume kube-api-access-2zbrk for pod openshift-network-diagnostics/network-check-target-9g22p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 22:57:41.912100 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:41.912020 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk podName:cd1248a5-5ddf-4647-b417-5099076f0133 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:57.912000511 +0000 UTC m=+34.384444673 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2zbrk" (UniqueName: "kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk") pod "network-check-target-9g22p" (UID: "cd1248a5-5ddf-4647-b417-5099076f0133") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 22:57:42.127213 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:42.127127 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:42.127612 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:42.127256 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:43.126900 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:43.126831 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:43.127107 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:43.126834 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:43.127107 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:43.126994 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:43.127107 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:43.127057 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:44.127188 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:44.127149 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:44.127713 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:44.127268 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:45.126849 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:45.126809 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:45.127027 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:45.126821 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:45.127027 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:45.126961 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:45.127151 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:45.127042 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:46.127125 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.127083 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:46.127844 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:46.127231 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:46.336043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.335775 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lzdpn" event={"ID":"d85e5451-3e8a-46bb-beb6-883353da3494","Type":"ContainerStarted","Data":"9ffb73b0f22a53745fa6e1afab5370c6389c5aec06efd60f6f6da35b48704867"} Jul 31 22:57:46.337442 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.337404 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" event={"ID":"e82a937d-4b34-4e32-8c73-76c56758aa66","Type":"ContainerStarted","Data":"4dc0594e6ee7e4cced87a3de8b603cc05cd5cbe5f145e542efda670d5f02d7c3"} Jul 31 22:57:46.338878 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.338857 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" event={"ID":"6e72564a-f37c-4507-893d-093ec9898e6b","Type":"ContainerStarted","Data":"ca13649f63e7474eacad4b824f3442d3232558629a1773a581fb5bee332330bb"} Jul 31 22:57:46.340471 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.340442 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sv97r" event={"ID":"02a7725c-7910-4311-bba9-dacdbc0d7ce5","Type":"ContainerStarted","Data":"af694f31830e11241949657a26da4e92d8d6231d92a240709c7cb361b947598d"} Jul 31 22:57:46.342700 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.342674 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" event={"ID":"815265b8-7f60-4832-81ce-d3f516f7b7c5","Type":"ContainerStarted","Data":"f175b61cc78d2f546a97c80ec43e24dbb1db692cb78f0e2ce00a4de8bb1ab2f7"} Jul 31 22:57:46.342791 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.342754 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" event={"ID":"815265b8-7f60-4832-81ce-d3f516f7b7c5","Type":"ContainerStarted","Data":"073c9f294c581540f331559949a2ae4dd1ab122c331d4800892f7d3b06e0fb8f"} Jul 31 22:57:46.342842 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.342798 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" event={"ID":"815265b8-7f60-4832-81ce-d3f516f7b7c5","Type":"ContainerStarted","Data":"a9da2bc3fcd7d197ef78feb02ed257f48574e2e388ef26418271937476716631"} Jul 31 22:57:46.343936 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.343918 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dqrvj" event={"ID":"2bd27fa7-9907-419d-af3b-43b77074de67","Type":"ContainerStarted","Data":"6d1d347b1d5143ff06023d3c8c239b3c9fd4e61a12d156845532ba51b41ecc30"} Jul 31 22:57:46.345239 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.345212 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5t5xj" event={"ID":"5753fb43-dec6-43b9-806e-3c593edb98dd","Type":"ContainerStarted","Data":"d8f5611430ce748db61b8f4f86e70aa6d920cde30e53572264f4bd1808cdf034"} Jul 31 22:57:46.346630 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.346610 2561 generic.go:356] "Generic (PLEG): container finished" podID="4c51d9ed-66ce-4160-8f08-bc20664c47b5" containerID="8f508e66ee2dcc06dc859491627bb772afae16e0c0a897b4e5b098a2ea050c7b" exitCode=0 Jul 31 22:57:46.346704 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.346637 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6gvt" event={"ID":"4c51d9ed-66ce-4160-8f08-bc20664c47b5","Type":"ContainerDied","Data":"8f508e66ee2dcc06dc859491627bb772afae16e0c0a897b4e5b098a2ea050c7b"} Jul 31 22:57:46.352027 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.351989 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lzdpn" podStartSLOduration=3.9018369379999998 podStartE2EDuration="22.351977733s" totalImagesPullingTime="18.450140795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:46.351845275 +0000 UTC m=+22.824289455" watchObservedRunningTime="2026-07-31 22:57:46.351977733 +0000 UTC m=+22.824421916" Jul 31 22:57:46.372324 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.372281 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wbqzc" podStartSLOduration=3.890961343 podStartE2EDuration="22.372265631s" totalImagesPullingTime="18.481304288s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:46.372089641 +0000 UTC m=+22.844533824" watchObservedRunningTime="2026-07-31 22:57:46.372265631 +0000 UTC m=+22.844709810" Jul 31 22:57:46.393322 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.393270 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-sv97r" podStartSLOduration=3.884424513 podStartE2EDuration="22.393251333s" totalImagesPullingTime="18.50882682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:46.393192581 +0000 UTC m=+22.865636764" watchObservedRunningTime="2026-07-31 22:57:46.393251333 +0000 UTC m=+22.865695518" Jul 31 22:57:46.424475 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.424432 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dqrvj" podStartSLOduration=7.636917418 podStartE2EDuration="22.424416177s" totalImagesPullingTime="14.787498759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:46.42397438 +0000 UTC m=+22.896418562" watchObservedRunningTime="2026-07-31 22:57:46.424416177 +0000 UTC m=+22.896860404" Jul 31 22:57:46.424616 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.424589 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5t5xj" podStartSLOduration=3.972444001 podStartE2EDuration="22.42458431s" totalImagesPullingTime="18.452140309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:46.408672438 +0000 UTC m=+22.881116621" watchObservedRunningTime="2026-07-31 22:57:46.42458431 +0000 UTC m=+22.897028493" Jul 31 22:57:46.461731 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.461705 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:46.546079 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.546054 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:46.600922 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.600897 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dqrvj" Jul 31 22:57:46.601485 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.601466 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dqrvj" Jul 31 22:57:46.626456 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:46.626320 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:47.126947 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:47.126912 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:47.127142 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:47.126913 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:47.127142 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:47.127048 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:47.127142 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:47.127124 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:47.349979 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:47.349899 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sjtn4" event={"ID":"23539586-9f8f-426a-b2a0-e9dbd8a06f85","Type":"ContainerStarted","Data":"516ea937132c67fe2ab15f869bdbb2b88432f6a42aad98371f3eb9ddbcb234ea"} Jul 31 22:57:47.353023 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:47.352986 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" event={"ID":"815265b8-7f60-4832-81ce-d3f516f7b7c5","Type":"ContainerStarted","Data":"bc1db9ea652a6b728605fa8531292144347b0337d45a70d51367022da715a3e5"} Jul 31 22:57:47.353182 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:47.353075 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" event={"ID":"815265b8-7f60-4832-81ce-d3f516f7b7c5","Type":"ContainerStarted","Data":"7b35c2aa50cd225a56f39f6b734845dd4bceab83ce0993d3350a2af9c2576c2d"} Jul 31 22:57:47.353182 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:47.353131 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" event={"ID":"815265b8-7f60-4832-81ce-d3f516f7b7c5","Type":"ContainerStarted","Data":"c4288111fe0d92cf3892f329c618a995087a01dbc27d266f550ed4f578a098d2"} Jul 31 22:57:47.353745 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:47.353714 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dqrvj" Jul 31 22:57:47.354149 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:47.354136 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dqrvj" Jul 31 22:57:47.365853 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:47.365805 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sjtn4" podStartSLOduration=4.9081757150000005 podStartE2EDuration="23.365790759s" totalImagesPullingTime="18.457615044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:47.365549827 +0000 UTC m=+23.837994007" watchObservedRunningTime="2026-07-31 22:57:47.365790759 +0000 UTC m=+23.838234942" Jul 31 22:57:47.488793 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:47.488738 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:47.525395 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:47.525370 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 31 22:57:47.619266 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:47.619192 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:47.682039 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:47.682006 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:48.012992 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:48.012849 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-31T22:57:47.525387362Z","UUID":"b8c67e13-f25b-482f-a501-eaf1a2f9e7b2","Handler":null,"Name":"","Endpoint":""} Jul 31 22:57:48.014253 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:48.014234 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 31 22:57:48.014375 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:48.014279 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 31 22:57:48.127076 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:48.127010 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:48.127673 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:48.127137 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:48.249250 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:48.249220 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:48.342204 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:48.342127 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:48.357583 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:48.357537 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" event={"ID":"6e72564a-f37c-4507-893d-093ec9898e6b","Type":"ContainerStarted","Data":"e3de615251cebe3727da13d42bdeb7ce4d5f139fc43d6918e8f4d25b7672f2b7"} Jul 31 22:57:48.538578 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:48.538536 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:48.621014 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:48.620928 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:48.695182 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:48.695158 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:49.112230 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:49.112151 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:49.126210 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:49.126180 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:49.126372 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:49.126179 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:49.126372 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:49.126312 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:49.126479 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:49.126377 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:49.185777 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:49.185740 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:49.255367 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:49.255323 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:49.363098 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:49.362982 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" event={"ID":"6e72564a-f37c-4507-893d-093ec9898e6b","Type":"ContainerStarted","Data":"1395d94a4d947d52f8f224b9fc1215e650ec2b37716e169d15fa86995075a824"} Jul 31 22:57:49.366816 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:49.366780 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" event={"ID":"815265b8-7f60-4832-81ce-d3f516f7b7c5","Type":"ContainerStarted","Data":"8fa57f2c75abf214b0281c6f5f001e4ab06f03449cd8c427422ae57dd6599362"} Jul 31 22:57:49.394994 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:49.394960 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:49.470274 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:49.470221 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:49.885583 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:49.885552 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5t5xj_5753fb43-dec6-43b9-806e-3c593edb98dd/dns-node-resolver/0.log" Jul 31 22:57:50.126617 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:50.126583 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:50.126805 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:50.126756 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:51.036044 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:51.035796 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:51.089078 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:51.089051 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lzdpn_d85e5451-3e8a-46bb-beb6-883353da3494/node-ca/0.log" Jul 31 22:57:51.110733 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:51.110706 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:51.127094 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:51.127012 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:51.127200 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:51.127023 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:51.127200 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:51.127174 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:51.127294 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:51.127275 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:51.375457 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:51.375423 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" event={"ID":"815265b8-7f60-4832-81ce-d3f516f7b7c5","Type":"ContainerStarted","Data":"41f7995348e37ba4557d224dfeaf3d17ba8bbc59e5c1617d0de1cc669542db40"} Jul 31 22:57:51.376304 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:51.376131 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:51.376304 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:51.376202 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:51.377706 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:51.377198 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6gvt" event={"ID":"4c51d9ed-66ce-4160-8f08-bc20664c47b5","Type":"ContainerStarted","Data":"3181a552f79c1beb73d8c34b2781d4dcebe30a8c57668e5506cb1424e17f97d4"} Jul 31 22:57:51.391255 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:51.391236 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:51.406169 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:51.406130 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" podStartSLOduration=8.883206021 podStartE2EDuration="27.406118627s" totalImagesPullingTime="18.522912606s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:51.404820534 +0000 UTC m=+27.877264742" watchObservedRunningTime="2026-07-31 22:57:51.406118627 +0000 UTC m=+27.878562844" Jul 31 22:57:51.406314 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:51.406297 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtfcb" podStartSLOduration=6.425035803 podStartE2EDuration="27.406291745s" totalImagesPullingTime="20.981255942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:57:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:49.390464701 +0000 UTC m=+25.862908884" watchObservedRunningTime="2026-07-31 22:57:51.406291745 +0000 UTC m=+27.878735979" Jul 31 22:57:52.127232 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:52.127154 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:52.127989 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:52.127275 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:52.187702 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:52.187658 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:52.381836 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:52.381749 2561 generic.go:356] "Generic (PLEG): container finished" podID="4c51d9ed-66ce-4160-8f08-bc20664c47b5" containerID="3181a552f79c1beb73d8c34b2781d4dcebe30a8c57668e5506cb1424e17f97d4" exitCode=0 Jul 31 22:57:52.381836 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:52.381799 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6gvt" event={"ID":"4c51d9ed-66ce-4160-8f08-bc20664c47b5","Type":"ContainerDied","Data":"3181a552f79c1beb73d8c34b2781d4dcebe30a8c57668e5506cb1424e17f97d4"} Jul 31 22:57:52.382426 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:52.382403 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:52.401753 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:52.401726 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:57:53.126807 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:53.126591 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:53.127075 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:53.126637 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:53.127075 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:53.126922 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:53.127075 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:53.126946 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:53.444175 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:53.444141 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lqgnq"] Jul 31 22:57:53.444645 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:53.444272 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:53.444645 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:53.444395 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:53.447656 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:53.447632 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9g22p"] Jul 31 22:57:53.447782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:53.447744 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:53.447863 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:53.447843 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:53.448435 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:53.448411 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-44zk2"] Jul 31 22:57:53.448559 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:53.448518 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:53.448649 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:53.448622 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:54.388117 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:54.388080 2561 generic.go:356] "Generic (PLEG): container finished" podID="4c51d9ed-66ce-4160-8f08-bc20664c47b5" containerID="b28c04699488a0b83053b95a2082e6a22b6d00f678cbaf12b563a9e00dd8fb91" exitCode=0 Jul 31 22:57:54.388303 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:54.388159 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6gvt" event={"ID":"4c51d9ed-66ce-4160-8f08-bc20664c47b5","Type":"ContainerDied","Data":"b28c04699488a0b83053b95a2082e6a22b6d00f678cbaf12b563a9e00dd8fb91"} Jul 31 22:57:54.588617 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:54.588581 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:54.732557 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:54.732476 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:54.839492 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:54.839451 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:54.928957 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:54.928880 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:55.126761 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:55.126731 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:55.126916 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:55.126730 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:55.126916 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:55.126845 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:55.126916 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:55.126739 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:55.127060 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:55.126924 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:55.127060 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:55.127010 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:55.451617 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:55.450728 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:55.696628 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:55.696370 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:56.291267 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:56.291232 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:56.394936 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:56.394902 2561 generic.go:356] "Generic (PLEG): container finished" podID="4c51d9ed-66ce-4160-8f08-bc20664c47b5" containerID="8c18d51c668674b08eac7aede6bbdd95c6747e59e6f6c253b96da429a76e3311" exitCode=0 Jul 31 22:57:56.395102 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:56.394966 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6gvt" event={"ID":"4c51d9ed-66ce-4160-8f08-bc20664c47b5","Type":"ContainerDied","Data":"8c18d51c668674b08eac7aede6bbdd95c6747e59e6f6c253b96da429a76e3311"} Jul 31 22:57:56.487858 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:56.487827 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:56.882155 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:56.882121 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:57.087335 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:57.087303 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:57.126429 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:57.126393 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:57.126606 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:57.126393 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:57.126606 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:57.126520 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:57.126745 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:57.126393 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:57.126745 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:57.126623 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:57.126745 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:57.126728 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:57.691590 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:57.691558 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:57.822102 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:57.822065 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:57.822284 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:57.822143 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs\") pod \"network-metrics-daemon-44zk2\" (UID: \"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e\") " pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:57.822284 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:57.822234 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:57:57.822284 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:57.822237 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 22:57:57.822410 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:57.822300 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs podName:58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e nodeName:}" failed. No retries permitted until 2026-07-31 22:58:29.822282164 +0000 UTC m=+66.294726331 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs") pod "network-metrics-daemon-44zk2" (UID: "58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:57:57.822410 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:57.822318 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret podName:40c30d12-8f96-4d63-a17e-5a3c101f2439 nodeName:}" failed. No retries permitted until 2026-07-31 22:58:29.822309253 +0000 UTC m=+66.294753418 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret") pod "global-pull-secret-syncer-lqgnq" (UID: "40c30d12-8f96-4d63-a17e-5a3c101f2439") : object "kube-system"/"original-pull-secret" not registered Jul 31 22:57:57.923454 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:57.923420 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zbrk\" (UniqueName: \"kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk\") pod \"network-check-target-9g22p\" (UID: \"cd1248a5-5ddf-4647-b417-5099076f0133\") " pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:57.923826 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:57.923600 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 22:57:57.923826 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:57.923617 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 22:57:57.923826 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:57.923626 2561 projected.go:196] Error preparing data for projected volume kube-api-access-2zbrk for pod openshift-network-diagnostics/network-check-target-9g22p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 22:57:57.923826 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:57.923717 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk podName:cd1248a5-5ddf-4647-b417-5099076f0133 nodeName:}" failed. No retries permitted until 2026-07-31 22:58:29.923702544 +0000 UTC m=+66.396146704 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2zbrk" (UniqueName: "kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk") pod "network-check-target-9g22p" (UID: "cd1248a5-5ddf-4647-b417-5099076f0133") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 22:57:58.089945 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:58.089901 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:58.299000 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:58.298957 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:58.610062 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:58.610027 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:58.704332 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:58.704301 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:59.127047 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:59.127015 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:57:59.127047 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:59.127035 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:57:59.127710 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:59.127037 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:57:59.127710 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:59.127136 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:57:59.127710 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:59.127216 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:57:59.127710 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:57:59.127263 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:57:59.301408 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:59.301375 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:59.374731 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:59.374691 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:59.891933 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:57:59.891897 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:00.109403 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:00.109371 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:00.487195 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:00.487167 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:00.683000 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:00.682916 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:00.796822 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:00.796790 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:00.897674 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:00.897644 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:01.126362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:01.126331 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:58:01.126362 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:01.126331 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:58:01.126635 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:01.126337 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:58:01.126635 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:01.126459 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:58:01.126635 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:01.126497 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:58:01.126635 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:01.126578 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:58:01.384743 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:01.384619 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:01.413653 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:01.413609 2561 generic.go:356] "Generic (PLEG): container finished" podID="4c51d9ed-66ce-4160-8f08-bc20664c47b5" containerID="d550c71db51a650c99f0c10e5fb117f87ba44083e1d04a27ddc4fb93ad3e45bd" exitCode=0 Jul 31 22:58:01.413810 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:01.413662 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6gvt" event={"ID":"4c51d9ed-66ce-4160-8f08-bc20664c47b5","Type":"ContainerDied","Data":"d550c71db51a650c99f0c10e5fb117f87ba44083e1d04a27ddc4fb93ad3e45bd"} Jul 31 22:58:01.679049 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:01.678966 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:01.882141 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:01.882110 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:02.281717 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:02.281682 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:02.419272 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:02.419241 2561 generic.go:356] "Generic (PLEG): container finished" podID="4c51d9ed-66ce-4160-8f08-bc20664c47b5" containerID="aed3af35f3ef18b98705feb6773667b71f0a5614b78fbe3a6e2f7a816f12f9a3" exitCode=0 Jul 31 22:58:02.419451 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:02.419292 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6gvt" event={"ID":"4c51d9ed-66ce-4160-8f08-bc20664c47b5","Type":"ContainerDied","Data":"aed3af35f3ef18b98705feb6773667b71f0a5614b78fbe3a6e2f7a816f12f9a3"} Jul 31 22:58:02.485087 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:02.485060 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:02.692210 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:02.692174 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:02.884517 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:02.884467 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:02.985326 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:02.985245 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:03.085404 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:03.085376 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:03.126473 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:03.126442 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:58:03.126600 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:03.126495 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:58:03.126600 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:03.126531 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:58:03.126685 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:03.126619 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:58:03.126685 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:03.126662 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:58:03.126783 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:03.126698 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:58:03.425575 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:03.425541 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6gvt" event={"ID":"4c51d9ed-66ce-4160-8f08-bc20664c47b5","Type":"ContainerStarted","Data":"e2df35e7c51da601c9444b291633cf89361848b947f8e60f384f8c475ce2585a"} Jul 31 22:58:03.451186 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:03.451134 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p6gvt" podStartSLOduration=10.218282667 podStartE2EDuration="39.45111748s" totalImagesPullingTime="29.232834813s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:57:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:03.449565431 +0000 UTC m=+39.922009613" watchObservedRunningTime="2026-07-31 22:58:03.45111748 +0000 UTC m=+39.923561662" Jul 31 22:58:03.663798 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:03.663761 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:04.089229 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:04.089192 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:04.281562 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:04.281528 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:04.492712 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:04.492627 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:04.684079 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:04.684045 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:04.880261 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:04.880227 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:04.985879 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:04.985847 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:05.127040 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:05.127004 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:58:05.127423 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:05.127004 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:58:05.127423 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:05.127114 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:58:05.127423 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:05.127196 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:58:05.127423 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:05.127004 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:58:05.127423 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:05.127279 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:58:05.279330 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:05.279244 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:07.127874 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:07.127835 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:58:07.127874 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:07.127880 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:58:07.128296 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:07.127941 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:58:07.128296 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:07.127954 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:58:07.128296 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:07.128029 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:58:07.128296 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:07.128068 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:58:09.126278 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:09.126238 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:58:09.126767 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:09.126239 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:58:09.126767 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:09.126370 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:58:09.126767 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:09.126239 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:58:09.126767 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:09.126443 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:58:09.126767 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:09.126497 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:58:11.126790 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:11.126754 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:58:11.126790 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:11.126779 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:58:11.127224 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:11.126850 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:58:11.127224 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:11.126864 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:58:11.127224 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:11.126912 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:58:11.127224 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:11.126975 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:58:13.126840 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:13.126788 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:58:13.126840 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:13.126835 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:58:13.127383 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:13.126836 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:58:13.127383 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:13.126938 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44zk2" podUID="58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e" Jul 31 22:58:13.127383 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:13.126986 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9g22p" podUID="cd1248a5-5ddf-4647-b417-5099076f0133" Jul 31 22:58:13.127383 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:13.127051 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lqgnq" podUID="40c30d12-8f96-4d63-a17e-5a3c101f2439" Jul 31 22:58:13.663473 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:13.663440 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:13.750688 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:13.750655 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:13.850782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:13.850752 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:13.949876 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:13.949794 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:14.050354 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.050322 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:14.842353 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.842324 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeReady" Jul 31 22:58:14.842736 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.842462 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 31 22:58:14.906420 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.906386 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zc76w"] Jul 31 22:58:14.931626 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.931596 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-prsgk"] Jul 31 22:58:14.931775 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.931729 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:14.934271 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.934245 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 31 22:58:14.934425 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.934276 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 31 22:58:14.934425 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.934282 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7jxx7\"" Jul 31 22:58:14.955614 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.955586 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-prsgk"] Jul 31 22:58:14.955614 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.955619 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zc76w"] Jul 31 22:58:14.955791 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.955693 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-prsgk" Jul 31 22:58:14.958442 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.958396 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c9ph2\"" Jul 31 22:58:14.958442 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.958396 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 31 22:58:14.958619 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.958452 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 31 22:58:14.958740 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:14.958696 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 31 22:58:15.005613 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.005579 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-km7l9"] Jul 31 22:58:15.027517 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.027473 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.029879 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.029843 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-km7l9"] Jul 31 22:58:15.030856 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.030830 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 31 22:58:15.030992 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.030830 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 31 22:58:15.030992 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.030878 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 31 22:58:15.031120 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.031011 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kmjpj\"" Jul 31 22:58:15.031120 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.031103 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 31 22:58:15.044671 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.044641 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b266c17f-9ecf-42a8-8843-4699f8a0cca5-config-volume\") pod \"dns-default-zc76w\" (UID: \"b266c17f-9ecf-42a8-8843-4699f8a0cca5\") " pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:15.044826 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.044677 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b266c17f-9ecf-42a8-8843-4699f8a0cca5-metrics-tls\") pod \"dns-default-zc76w\" (UID: \"b266c17f-9ecf-42a8-8843-4699f8a0cca5\") " pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:15.044826 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.044720 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f85hn\" (UniqueName: \"kubernetes.io/projected/b266c17f-9ecf-42a8-8843-4699f8a0cca5-kube-api-access-f85hn\") pod \"dns-default-zc76w\" (UID: \"b266c17f-9ecf-42a8-8843-4699f8a0cca5\") " pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:15.044826 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.044779 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b266c17f-9ecf-42a8-8843-4699f8a0cca5-tmp-dir\") pod \"dns-default-zc76w\" (UID: \"b266c17f-9ecf-42a8-8843-4699f8a0cca5\") " pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:15.054613 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.054584 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:15.126266 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.126184 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:58:15.126387 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.126292 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:58:15.126387 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.126320 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:58:15.129714 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.129683 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 31 22:58:15.129714 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.129704 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 31 22:58:15.129905 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.129689 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h8t45\"" Jul 31 22:58:15.129905 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.129744 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jtx5g\"" Jul 31 22:58:15.129905 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.129763 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 31 22:58:15.129905 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.129725 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 31 22:58:15.145242 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.145219 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a18cb43c-48f7-4bb1-853a-ad278519677f-crio-socket\") pod \"insights-runtime-extractor-km7l9\" (UID: \"a18cb43c-48f7-4bb1-853a-ad278519677f\") " pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.145343 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.145254 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b266c17f-9ecf-42a8-8843-4699f8a0cca5-config-volume\") pod \"dns-default-zc76w\" (UID: \"b266c17f-9ecf-42a8-8843-4699f8a0cca5\") " pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:15.145343 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.145278 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b266c17f-9ecf-42a8-8843-4699f8a0cca5-metrics-tls\") pod \"dns-default-zc76w\" (UID: \"b266c17f-9ecf-42a8-8843-4699f8a0cca5\") " pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:15.145343 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.145300 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a18cb43c-48f7-4bb1-853a-ad278519677f-data-volume\") pod \"insights-runtime-extractor-km7l9\" (UID: \"a18cb43c-48f7-4bb1-853a-ad278519677f\") " pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.145343 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.145318 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7ghq\" (UniqueName: \"kubernetes.io/projected/afb3784e-7558-4f93-9633-3423b9cbf971-kube-api-access-s7ghq\") pod \"ingress-canary-prsgk\" (UID: \"afb3784e-7558-4f93-9633-3423b9cbf971\") " pod="openshift-ingress-canary/ingress-canary-prsgk" Jul 31 22:58:15.145343 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.145338 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a18cb43c-48f7-4bb1-853a-ad278519677f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-km7l9\" (UID: \"a18cb43c-48f7-4bb1-853a-ad278519677f\") " pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.145556 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.145374 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f85hn\" (UniqueName: \"kubernetes.io/projected/b266c17f-9ecf-42a8-8843-4699f8a0cca5-kube-api-access-f85hn\") pod \"dns-default-zc76w\" (UID: \"b266c17f-9ecf-42a8-8843-4699f8a0cca5\") " pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:15.145556 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.145393 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a18cb43c-48f7-4bb1-853a-ad278519677f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-km7l9\" (UID: \"a18cb43c-48f7-4bb1-853a-ad278519677f\") " pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.145556 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.145410 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhlx5\" (UniqueName: \"kubernetes.io/projected/a18cb43c-48f7-4bb1-853a-ad278519677f-kube-api-access-nhlx5\") pod \"insights-runtime-extractor-km7l9\" (UID: \"a18cb43c-48f7-4bb1-853a-ad278519677f\") " pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.145556 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.145425 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/afb3784e-7558-4f93-9633-3423b9cbf971-cert\") pod \"ingress-canary-prsgk\" (UID: \"afb3784e-7558-4f93-9633-3423b9cbf971\") " pod="openshift-ingress-canary/ingress-canary-prsgk" Jul 31 22:58:15.145556 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.145552 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b266c17f-9ecf-42a8-8843-4699f8a0cca5-tmp-dir\") pod \"dns-default-zc76w\" (UID: \"b266c17f-9ecf-42a8-8843-4699f8a0cca5\") " pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:15.145814 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.145800 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b266c17f-9ecf-42a8-8843-4699f8a0cca5-tmp-dir\") pod \"dns-default-zc76w\" (UID: \"b266c17f-9ecf-42a8-8843-4699f8a0cca5\") " pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:15.145851 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.145813 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b266c17f-9ecf-42a8-8843-4699f8a0cca5-config-volume\") pod \"dns-default-zc76w\" (UID: \"b266c17f-9ecf-42a8-8843-4699f8a0cca5\") " pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:15.149249 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.149231 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b266c17f-9ecf-42a8-8843-4699f8a0cca5-metrics-tls\") pod \"dns-default-zc76w\" (UID: \"b266c17f-9ecf-42a8-8843-4699f8a0cca5\") " pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:15.153994 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.153972 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f85hn\" (UniqueName: \"kubernetes.io/projected/b266c17f-9ecf-42a8-8843-4699f8a0cca5-kube-api-access-f85hn\") pod \"dns-default-zc76w\" (UID: \"b266c17f-9ecf-42a8-8843-4699f8a0cca5\") " pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:15.240578 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.240539 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:15.246750 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.246719 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a18cb43c-48f7-4bb1-853a-ad278519677f-crio-socket\") pod \"insights-runtime-extractor-km7l9\" (UID: \"a18cb43c-48f7-4bb1-853a-ad278519677f\") " pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.246882 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.246773 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a18cb43c-48f7-4bb1-853a-ad278519677f-data-volume\") pod \"insights-runtime-extractor-km7l9\" (UID: \"a18cb43c-48f7-4bb1-853a-ad278519677f\") " pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.246882 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.246796 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7ghq\" (UniqueName: \"kubernetes.io/projected/afb3784e-7558-4f93-9633-3423b9cbf971-kube-api-access-s7ghq\") pod \"ingress-canary-prsgk\" (UID: \"afb3784e-7558-4f93-9633-3423b9cbf971\") " pod="openshift-ingress-canary/ingress-canary-prsgk" Jul 31 22:58:15.246882 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.246820 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a18cb43c-48f7-4bb1-853a-ad278519677f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-km7l9\" (UID: \"a18cb43c-48f7-4bb1-853a-ad278519677f\") " pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.246882 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.246861 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a18cb43c-48f7-4bb1-853a-ad278519677f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-km7l9\" (UID: \"a18cb43c-48f7-4bb1-853a-ad278519677f\") " pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.247133 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.246884 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhlx5\" (UniqueName: \"kubernetes.io/projected/a18cb43c-48f7-4bb1-853a-ad278519677f-kube-api-access-nhlx5\") pod \"insights-runtime-extractor-km7l9\" (UID: \"a18cb43c-48f7-4bb1-853a-ad278519677f\") " pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.247133 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.246906 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/afb3784e-7558-4f93-9633-3423b9cbf971-cert\") pod \"ingress-canary-prsgk\" (UID: \"afb3784e-7558-4f93-9633-3423b9cbf971\") " pod="openshift-ingress-canary/ingress-canary-prsgk" Jul 31 22:58:15.247133 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.246966 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a18cb43c-48f7-4bb1-853a-ad278519677f-crio-socket\") pod \"insights-runtime-extractor-km7l9\" (UID: \"a18cb43c-48f7-4bb1-853a-ad278519677f\") " pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.247280 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.247159 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a18cb43c-48f7-4bb1-853a-ad278519677f-data-volume\") pod \"insights-runtime-extractor-km7l9\" (UID: \"a18cb43c-48f7-4bb1-853a-ad278519677f\") " pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.247447 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.247419 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a18cb43c-48f7-4bb1-853a-ad278519677f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-km7l9\" (UID: \"a18cb43c-48f7-4bb1-853a-ad278519677f\") " pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.249199 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.249181 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a18cb43c-48f7-4bb1-853a-ad278519677f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-km7l9\" (UID: \"a18cb43c-48f7-4bb1-853a-ad278519677f\") " pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.249358 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.249342 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/afb3784e-7558-4f93-9633-3423b9cbf971-cert\") pod \"ingress-canary-prsgk\" (UID: \"afb3784e-7558-4f93-9633-3423b9cbf971\") " pod="openshift-ingress-canary/ingress-canary-prsgk" Jul 31 22:58:15.255304 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.255278 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhlx5\" (UniqueName: \"kubernetes.io/projected/a18cb43c-48f7-4bb1-853a-ad278519677f-kube-api-access-nhlx5\") pod \"insights-runtime-extractor-km7l9\" (UID: \"a18cb43c-48f7-4bb1-853a-ad278519677f\") " pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.255412 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.255374 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7ghq\" (UniqueName: \"kubernetes.io/projected/afb3784e-7558-4f93-9633-3423b9cbf971-kube-api-access-s7ghq\") pod \"ingress-canary-prsgk\" (UID: \"afb3784e-7558-4f93-9633-3423b9cbf971\") " pod="openshift-ingress-canary/ingress-canary-prsgk" Jul 31 22:58:15.264167 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.264145 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-prsgk" Jul 31 22:58:15.337519 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.337471 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-km7l9" Jul 31 22:58:15.479570 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.479473 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zc76w"] Jul 31 22:58:15.481088 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:15.481057 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb266c17f_9ecf_42a8_8843_4699f8a0cca5.slice/crio-90bbadbb967c8e028bfd4f4b21dc9bb82b54ae10900050c065e325fd8aa5c666 WatchSource:0}: Error finding container 90bbadbb967c8e028bfd4f4b21dc9bb82b54ae10900050c065e325fd8aa5c666: Status 404 returned error can't find the container with id 90bbadbb967c8e028bfd4f4b21dc9bb82b54ae10900050c065e325fd8aa5c666 Jul 31 22:58:15.485982 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.485960 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-prsgk"] Jul 31 22:58:15.488908 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:15.488877 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafb3784e_7558_4f93_9633_3423b9cbf971.slice/crio-46bb018f58dd17b74119f46fb7f7fa68068f0ec3f6a44bb2718ac570dbb4b612 WatchSource:0}: Error finding container 46bb018f58dd17b74119f46fb7f7fa68068f0ec3f6a44bb2718ac570dbb4b612: Status 404 returned error can't find the container with id 46bb018f58dd17b74119f46fb7f7fa68068f0ec3f6a44bb2718ac570dbb4b612 Jul 31 22:58:15.541428 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.541398 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-km7l9"] Jul 31 22:58:15.544494 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:15.544467 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18cb43c_48f7_4bb1_853a_ad278519677f.slice/crio-7e6a010de1ed710bc97804d0f02cf2fc61df51d1d8a349c353c466ef653f34b0 WatchSource:0}: Error finding container 7e6a010de1ed710bc97804d0f02cf2fc61df51d1d8a349c353c466ef653f34b0: Status 404 returned error can't find the container with id 7e6a010de1ed710bc97804d0f02cf2fc61df51d1d8a349c353c466ef653f34b0 Jul 31 22:58:15.640766 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.640739 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:15.742250 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:15.742219 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:16.036424 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:16.036345 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:16.136785 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:16.136745 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:16.458276 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:16.458203 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zc76w" event={"ID":"b266c17f-9ecf-42a8-8843-4699f8a0cca5","Type":"ContainerStarted","Data":"90bbadbb967c8e028bfd4f4b21dc9bb82b54ae10900050c065e325fd8aa5c666"} Jul 31 22:58:16.459717 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:16.459685 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-prsgk" event={"ID":"afb3784e-7558-4f93-9633-3423b9cbf971","Type":"ContainerStarted","Data":"46bb018f58dd17b74119f46fb7f7fa68068f0ec3f6a44bb2718ac570dbb4b612"} Jul 31 22:58:16.461765 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:16.461736 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-km7l9" event={"ID":"a18cb43c-48f7-4bb1-853a-ad278519677f","Type":"ContainerStarted","Data":"6053eba2a7ec9d9d521418fc5f7a031a9dd99b15d543879ffdbeb1de24c9682c"} Jul 31 22:58:16.461876 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:16.461795 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-km7l9" event={"ID":"a18cb43c-48f7-4bb1-853a-ad278519677f","Type":"ContainerStarted","Data":"7e6a010de1ed710bc97804d0f02cf2fc61df51d1d8a349c353c466ef653f34b0"} Jul 31 22:58:17.465888 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:17.465853 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-km7l9" event={"ID":"a18cb43c-48f7-4bb1-853a-ad278519677f","Type":"ContainerStarted","Data":"f7f0e754edaac6720283623639b8516562343fc4c0fbc9dc2cd3212695048374"} Jul 31 22:58:18.680553 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:18.680470 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:18.781028 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:18.780988 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:18.854172 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:18.854141 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jfsgj"] Jul 31 22:58:18.865709 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:18.865633 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:18.878614 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:18.878575 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jfsgj"] Jul 31 22:58:18.878783 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:18.878687 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jfsgj" Jul 31 22:58:18.881456 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:18.881427 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 31 22:58:18.881576 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:18.881474 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6zkbc\"" Jul 31 22:58:18.978259 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:18.978168 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d14db40d-3234-4eb9-b104-28fc2bae1b12-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-jfsgj\" (UID: \"d14db40d-3234-4eb9-b104-28fc2bae1b12\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jfsgj" Jul 31 22:58:19.066795 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:19.066762 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:19.078562 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:19.078531 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d14db40d-3234-4eb9-b104-28fc2bae1b12-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-jfsgj\" (UID: \"d14db40d-3234-4eb9-b104-28fc2bae1b12\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jfsgj" Jul 31 22:58:19.081176 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:19.081150 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d14db40d-3234-4eb9-b104-28fc2bae1b12-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-jfsgj\" (UID: \"d14db40d-3234-4eb9-b104-28fc2bae1b12\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jfsgj" Jul 31 22:58:19.187363 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:19.187318 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jfsgj" Jul 31 22:58:19.323131 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:19.323094 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jfsgj"] Jul 31 22:58:19.473762 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:19.473725 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zc76w" event={"ID":"b266c17f-9ecf-42a8-8843-4699f8a0cca5","Type":"ContainerStarted","Data":"69cae158f0ce4887615c2202efec0cf32e56d3d6558bb17e56eabe69d86f7050"} Jul 31 22:58:19.473942 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:19.473789 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zc76w" event={"ID":"b266c17f-9ecf-42a8-8843-4699f8a0cca5","Type":"ContainerStarted","Data":"80602c3533d93187683a304d9b2550b77a4be6457717993287a7fe0b6a1494e7"} Jul 31 22:58:19.473942 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:19.473819 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:19.475406 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:19.475376 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-prsgk" event={"ID":"afb3784e-7558-4f93-9633-3423b9cbf971","Type":"ContainerStarted","Data":"ec2d5c655c3dc71e81e70dbe4c07c9308d95f822fd62ef3f97964ccf730b1164"} Jul 31 22:58:19.512259 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:19.512147 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zc76w" podStartSLOduration=2.597788472 podStartE2EDuration="5.512127737s" totalImagesPullingTime="2.914339265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:58:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:19.511604029 +0000 UTC m=+55.984048234" watchObservedRunningTime="2026-07-31 22:58:19.512127737 +0000 UTC m=+55.984571921" Jul 31 22:58:19.536749 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:19.536690 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-prsgk" podStartSLOduration=2.340659453 podStartE2EDuration="5.536670966s" totalImagesPullingTime="3.196011513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:58:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:19.535878409 +0000 UTC m=+56.008322611" watchObservedRunningTime="2026-07-31 22:58:19.536670966 +0000 UTC m=+56.009115149" Jul 31 22:58:19.681462 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:19.681426 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd14db40d_3234_4eb9_b104_28fc2bae1b12.slice/crio-1aaae6962e609c0dad6d3353862ec4a1ccef8961c4e14a7930f25d6be1e8f22c WatchSource:0}: Error finding container 1aaae6962e609c0dad6d3353862ec4a1ccef8961c4e14a7930f25d6be1e8f22c: Status 404 returned error can't find the container with id 1aaae6962e609c0dad6d3353862ec4a1ccef8961c4e14a7930f25d6be1e8f22c Jul 31 22:58:20.483204 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:20.483170 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jfsgj" event={"ID":"d14db40d-3234-4eb9-b104-28fc2bae1b12","Type":"ContainerStarted","Data":"1aaae6962e609c0dad6d3353862ec4a1ccef8961c4e14a7930f25d6be1e8f22c"} Jul 31 22:58:20.485459 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:20.485423 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-km7l9" event={"ID":"a18cb43c-48f7-4bb1-853a-ad278519677f","Type":"ContainerStarted","Data":"23067290c5c5f0f99d1edfde8716963a7bfcec45ad06090668ff38d9d770a93f"} Jul 31 22:58:20.507240 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:20.507187 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-km7l9" podStartSLOduration=2.549821356 podStartE2EDuration="6.507163282s" totalImagesPullingTime="3.957341926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:58:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:20.505654676 +0000 UTC m=+56.978098859" watchObservedRunningTime="2026-07-31 22:58:20.507163282 +0000 UTC m=+56.979607469" Jul 31 22:58:21.153442 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.153408 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:21.253716 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.253687 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:21.490397 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.490302 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jfsgj" event={"ID":"d14db40d-3234-4eb9-b104-28fc2bae1b12","Type":"ContainerStarted","Data":"523924222e2deed56cde1694c9a613358b506356afa8f83510560c728a9d0a3e"} Jul 31 22:58:21.490710 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.490689 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jfsgj" Jul 31 22:58:21.495386 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.495357 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jfsgj" Jul 31 22:58:21.517062 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.517011 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jfsgj" podStartSLOduration=2.582030962 podStartE2EDuration="3.516993211s" totalImagesPullingTime="934.962249ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:58:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:21.515378611 +0000 UTC m=+57.987822798" watchObservedRunningTime="2026-07-31 22:58:21.516993211 +0000 UTC m=+57.989437393" Jul 31 22:58:21.879130 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.879096 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:21.916424 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.916395 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-vfxp5"] Jul 31 22:58:21.918238 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.918224 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" Jul 31 22:58:21.922195 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.922171 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 31 22:58:21.922318 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.922236 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 31 22:58:21.922318 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.922262 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 31 22:58:21.922318 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.922274 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 31 22:58:21.922318 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.922281 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 31 22:58:21.922318 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.922262 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-762kb\"" Jul 31 22:58:21.930361 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.930342 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-vfxp5"] Jul 31 22:58:21.954648 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:21.954619 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:22.000903 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.000866 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a4dee31d-eaee-4012-88bf-e7a6d1d759ad-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-vfxp5\" (UID: \"a4dee31d-eaee-4012-88bf-e7a6d1d759ad\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" Jul 31 22:58:22.001033 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.000907 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4dee31d-eaee-4012-88bf-e7a6d1d759ad-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-vfxp5\" (UID: \"a4dee31d-eaee-4012-88bf-e7a6d1d759ad\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" Jul 31 22:58:22.001033 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.000931 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpjdr\" (UniqueName: \"kubernetes.io/projected/a4dee31d-eaee-4012-88bf-e7a6d1d759ad-kube-api-access-lpjdr\") pod \"prometheus-operator-758d7cb874-vfxp5\" (UID: \"a4dee31d-eaee-4012-88bf-e7a6d1d759ad\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" Jul 31 22:58:22.001138 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.001025 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4dee31d-eaee-4012-88bf-e7a6d1d759ad-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-vfxp5\" (UID: \"a4dee31d-eaee-4012-88bf-e7a6d1d759ad\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" Jul 31 22:58:22.076216 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.076182 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:22.101490 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.101465 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a4dee31d-eaee-4012-88bf-e7a6d1d759ad-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-vfxp5\" (UID: \"a4dee31d-eaee-4012-88bf-e7a6d1d759ad\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" Jul 31 22:58:22.101614 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.101495 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4dee31d-eaee-4012-88bf-e7a6d1d759ad-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-vfxp5\" (UID: \"a4dee31d-eaee-4012-88bf-e7a6d1d759ad\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" Jul 31 22:58:22.101614 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.101532 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpjdr\" (UniqueName: \"kubernetes.io/projected/a4dee31d-eaee-4012-88bf-e7a6d1d759ad-kube-api-access-lpjdr\") pod \"prometheus-operator-758d7cb874-vfxp5\" (UID: \"a4dee31d-eaee-4012-88bf-e7a6d1d759ad\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" Jul 31 22:58:22.101721 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.101670 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4dee31d-eaee-4012-88bf-e7a6d1d759ad-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-vfxp5\" (UID: \"a4dee31d-eaee-4012-88bf-e7a6d1d759ad\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" Jul 31 22:58:22.102280 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.102256 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4dee31d-eaee-4012-88bf-e7a6d1d759ad-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-vfxp5\" (UID: \"a4dee31d-eaee-4012-88bf-e7a6d1d759ad\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" Jul 31 22:58:22.103888 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.103865 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4dee31d-eaee-4012-88bf-e7a6d1d759ad-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-vfxp5\" (UID: \"a4dee31d-eaee-4012-88bf-e7a6d1d759ad\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" Jul 31 22:58:22.103984 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.103968 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a4dee31d-eaee-4012-88bf-e7a6d1d759ad-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-vfxp5\" (UID: \"a4dee31d-eaee-4012-88bf-e7a6d1d759ad\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" Jul 31 22:58:22.109882 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.109861 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpjdr\" (UniqueName: \"kubernetes.io/projected/a4dee31d-eaee-4012-88bf-e7a6d1d759ad-kube-api-access-lpjdr\") pod \"prometheus-operator-758d7cb874-vfxp5\" (UID: \"a4dee31d-eaee-4012-88bf-e7a6d1d759ad\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" Jul 31 22:58:22.226987 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.226900 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" Jul 31 22:58:22.348009 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.347978 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-vfxp5"] Jul 31 22:58:22.350615 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:22.350589 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4dee31d_eaee_4012_88bf_e7a6d1d759ad.slice/crio-c1e8662a21e3736c41f7a493b70ad3719b12a6779d5c4ceb8453c26049c322a2 WatchSource:0}: Error finding container c1e8662a21e3736c41f7a493b70ad3719b12a6779d5c4ceb8453c26049c322a2: Status 404 returned error can't find the container with id c1e8662a21e3736c41f7a493b70ad3719b12a6779d5c4ceb8453c26049c322a2 Jul 31 22:58:22.496491 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.496406 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" event={"ID":"a4dee31d-eaee-4012-88bf-e7a6d1d759ad","Type":"ContainerStarted","Data":"c1e8662a21e3736c41f7a493b70ad3719b12a6779d5c4ceb8453c26049c322a2"} Jul 31 22:58:22.616084 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.616052 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:22.716938 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:22.716906 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:23.861457 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:23.861397 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:24.095115 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:24.095080 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:24.194017 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:24.193919 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:24.399521 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:24.399475 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlbsb" Jul 31 22:58:24.507620 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:24.507533 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" event={"ID":"a4dee31d-eaee-4012-88bf-e7a6d1d759ad","Type":"ContainerStarted","Data":"5b3f88ac59b0c83c1b84337a2a8b6ab0105219564540257713da474d34263d5e"} Jul 31 22:58:24.507620 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:24.507591 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" event={"ID":"a4dee31d-eaee-4012-88bf-e7a6d1d759ad","Type":"ContainerStarted","Data":"5e8985a4dbb0dfd986fbb29ed59b70f8a54fff0b73a37d870dbfffaa041d848d"} Jul 31 22:58:24.528917 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:24.528863 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-758d7cb874-vfxp5" podStartSLOduration=2.329660433 podStartE2EDuration="3.528842734s" totalImagesPullingTime="1.199182301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:58:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:24.528439802 +0000 UTC m=+61.000883986" watchObservedRunningTime="2026-07-31 22:58:24.528842734 +0000 UTC m=+61.001286917" Jul 31 22:58:24.540657 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:24.540629 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:24.625223 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:24.625186 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:26.093464 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.093430 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:26.476331 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.476250 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:26.564782 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.564748 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5"] Jul 31 22:58:26.567482 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.567464 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" Jul 31 22:58:26.571013 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.570990 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 31 22:58:26.571125 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.570998 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-knj9b\"" Jul 31 22:58:26.571406 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.571386 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 31 22:58:26.576134 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.576114 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv"] Jul 31 22:58:26.578159 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.578142 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.580638 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.580617 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 31 22:58:26.580729 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.580670 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-v4vl8\"" Jul 31 22:58:26.580814 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.580793 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 31 22:58:26.581392 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.581377 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 31 22:58:26.589646 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.589625 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5"] Jul 31 22:58:26.600623 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.600597 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pj2bq"] Jul 31 22:58:26.603018 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.603004 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.606520 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.606480 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 31 22:58:26.606520 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.606518 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 31 22:58:26.606687 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.606484 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kgbfl\"" Jul 31 22:58:26.606687 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.606655 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 31 22:58:26.616493 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.616470 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:26.623131 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.623107 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv"] Jul 31 22:58:26.631650 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.631627 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/189420ca-6274-47bb-b106-399b9002d242-sys\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.631780 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.631653 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/189420ca-6274-47bb-b106-399b9002d242-node-exporter-wtmp\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.631780 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.631673 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f6936db-6831-43bf-a6e4-fd33552c2657-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-zdfg5\" (UID: \"0f6936db-6831-43bf-a6e4-fd33552c2657\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" Jul 31 22:58:26.631780 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.631696 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/189420ca-6274-47bb-b106-399b9002d242-node-exporter-tls\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.631780 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.631735 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0689778d-1ccf-4e60-a5f0-72ad871a40ba-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.631949 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.631784 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0689778d-1ccf-4e60-a5f0-72ad871a40ba-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.631949 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.631842 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0689778d-1ccf-4e60-a5f0-72ad871a40ba-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.631949 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.631869 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jczp\" (UniqueName: \"kubernetes.io/projected/0689778d-1ccf-4e60-a5f0-72ad871a40ba-kube-api-access-8jczp\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.631949 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.631908 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/189420ca-6274-47bb-b106-399b9002d242-root\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.631949 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.631926 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0689778d-1ccf-4e60-a5f0-72ad871a40ba-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.631949 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.631944 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/189420ca-6274-47bb-b106-399b9002d242-metrics-client-ca\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.632226 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.632028 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/189420ca-6274-47bb-b106-399b9002d242-node-exporter-textfile\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.632226 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.632086 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0689778d-1ccf-4e60-a5f0-72ad871a40ba-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.632226 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.632117 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/189420ca-6274-47bb-b106-399b9002d242-node-exporter-accelerators-collector-config\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.632226 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.632151 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sf2s\" (UniqueName: \"kubernetes.io/projected/0f6936db-6831-43bf-a6e4-fd33552c2657-kube-api-access-5sf2s\") pod \"openshift-state-metrics-6bb4d7b4dd-zdfg5\" (UID: \"0f6936db-6831-43bf-a6e4-fd33552c2657\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" Jul 31 22:58:26.632226 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.632190 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0f6936db-6831-43bf-a6e4-fd33552c2657-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-zdfg5\" (UID: \"0f6936db-6831-43bf-a6e4-fd33552c2657\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" Jul 31 22:58:26.632226 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.632222 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f6936db-6831-43bf-a6e4-fd33552c2657-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-zdfg5\" (UID: \"0f6936db-6831-43bf-a6e4-fd33552c2657\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" Jul 31 22:58:26.632453 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.632259 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/189420ca-6274-47bb-b106-399b9002d242-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.632453 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.632285 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbzkn\" (UniqueName: \"kubernetes.io/projected/189420ca-6274-47bb-b106-399b9002d242-kube-api-access-tbzkn\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.731763 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.731680 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:26.732641 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.732620 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/189420ca-6274-47bb-b106-399b9002d242-root\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.732694 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.732651 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0689778d-1ccf-4e60-a5f0-72ad871a40ba-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.732694 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.732671 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/189420ca-6274-47bb-b106-399b9002d242-metrics-client-ca\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.732770 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.732731 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/189420ca-6274-47bb-b106-399b9002d242-root\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.732811 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.732771 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/189420ca-6274-47bb-b106-399b9002d242-node-exporter-textfile\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.732846 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.732819 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0689778d-1ccf-4e60-a5f0-72ad871a40ba-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.732888 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.732841 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/189420ca-6274-47bb-b106-399b9002d242-node-exporter-accelerators-collector-config\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.732888 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.732876 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sf2s\" (UniqueName: \"kubernetes.io/projected/0f6936db-6831-43bf-a6e4-fd33552c2657-kube-api-access-5sf2s\") pod \"openshift-state-metrics-6bb4d7b4dd-zdfg5\" (UID: \"0f6936db-6831-43bf-a6e4-fd33552c2657\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" Jul 31 22:58:26.732987 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.732907 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0f6936db-6831-43bf-a6e4-fd33552c2657-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-zdfg5\" (UID: \"0f6936db-6831-43bf-a6e4-fd33552c2657\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" Jul 31 22:58:26.732987 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:26.732957 2561 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 31 22:58:26.733075 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:26.733022 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0689778d-1ccf-4e60-a5f0-72ad871a40ba-kube-state-metrics-tls podName:0689778d-1ccf-4e60-a5f0-72ad871a40ba nodeName:}" failed. No retries permitted until 2026-07-31 22:58:27.233001383 +0000 UTC m=+63.705445552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/0689778d-1ccf-4e60-a5f0-72ad871a40ba-kube-state-metrics-tls") pod "kube-state-metrics-6bd9664c4-5fcmv" (UID: "0689778d-1ccf-4e60-a5f0-72ad871a40ba") : secret "kube-state-metrics-tls" not found Jul 31 22:58:26.733167 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733136 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/189420ca-6274-47bb-b106-399b9002d242-node-exporter-textfile\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.733238 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733200 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f6936db-6831-43bf-a6e4-fd33552c2657-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-zdfg5\" (UID: \"0f6936db-6831-43bf-a6e4-fd33552c2657\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" Jul 31 22:58:26.733296 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733259 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/189420ca-6274-47bb-b106-399b9002d242-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.733296 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733286 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbzkn\" (UniqueName: \"kubernetes.io/projected/189420ca-6274-47bb-b106-399b9002d242-kube-api-access-tbzkn\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.733401 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733323 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/189420ca-6274-47bb-b106-399b9002d242-sys\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.733401 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733343 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/189420ca-6274-47bb-b106-399b9002d242-metrics-client-ca\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.733401 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733349 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/189420ca-6274-47bb-b106-399b9002d242-node-exporter-wtmp\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.733561 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733407 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/189420ca-6274-47bb-b106-399b9002d242-sys\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.733561 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733413 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0689778d-1ccf-4e60-a5f0-72ad871a40ba-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.733561 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733414 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/189420ca-6274-47bb-b106-399b9002d242-node-exporter-accelerators-collector-config\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.733561 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733406 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f6936db-6831-43bf-a6e4-fd33552c2657-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-zdfg5\" (UID: \"0f6936db-6831-43bf-a6e4-fd33552c2657\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" Jul 31 22:58:26.733561 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733481 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/189420ca-6274-47bb-b106-399b9002d242-node-exporter-tls\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.733561 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733545 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0689778d-1ccf-4e60-a5f0-72ad871a40ba-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.733829 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733573 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0689778d-1ccf-4e60-a5f0-72ad871a40ba-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.733829 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733603 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0689778d-1ccf-4e60-a5f0-72ad871a40ba-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.733829 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733631 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jczp\" (UniqueName: \"kubernetes.io/projected/0689778d-1ccf-4e60-a5f0-72ad871a40ba-kube-api-access-8jczp\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.733829 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733720 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/189420ca-6274-47bb-b106-399b9002d242-node-exporter-wtmp\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.734036 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.733964 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f6936db-6831-43bf-a6e4-fd33552c2657-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-zdfg5\" (UID: \"0f6936db-6831-43bf-a6e4-fd33552c2657\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" Jul 31 22:58:26.734193 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.734169 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0689778d-1ccf-4e60-a5f0-72ad871a40ba-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.734468 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.734443 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0689778d-1ccf-4e60-a5f0-72ad871a40ba-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.735766 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.735733 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0f6936db-6831-43bf-a6e4-fd33552c2657-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-zdfg5\" (UID: \"0f6936db-6831-43bf-a6e4-fd33552c2657\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" Jul 31 22:58:26.735868 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.735841 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/189420ca-6274-47bb-b106-399b9002d242-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.736209 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.736189 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/189420ca-6274-47bb-b106-399b9002d242-node-exporter-tls\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.736264 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.736207 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f6936db-6831-43bf-a6e4-fd33552c2657-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-zdfg5\" (UID: \"0f6936db-6831-43bf-a6e4-fd33552c2657\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" Jul 31 22:58:26.736324 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.736271 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0689778d-1ccf-4e60-a5f0-72ad871a40ba-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.746069 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.746041 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbzkn\" (UniqueName: \"kubernetes.io/projected/189420ca-6274-47bb-b106-399b9002d242-kube-api-access-tbzkn\") pod \"node-exporter-pj2bq\" (UID: \"189420ca-6274-47bb-b106-399b9002d242\") " pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.746235 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.746212 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jczp\" (UniqueName: \"kubernetes.io/projected/0689778d-1ccf-4e60-a5f0-72ad871a40ba-kube-api-access-8jczp\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:26.746277 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.746215 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sf2s\" (UniqueName: \"kubernetes.io/projected/0f6936db-6831-43bf-a6e4-fd33552c2657-kube-api-access-5sf2s\") pod \"openshift-state-metrics-6bb4d7b4dd-zdfg5\" (UID: \"0f6936db-6831-43bf-a6e4-fd33552c2657\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" Jul 31 22:58:26.833939 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.833895 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:26.876544 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.876485 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" Jul 31 22:58:26.911457 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.911416 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pj2bq" Jul 31 22:58:26.923194 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:26.923162 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod189420ca_6274_47bb_b106_399b9002d242.slice/crio-777fc22726bc541f42bbc32522bb51f19c8dca4d4bc8d882c5d56f1473f2e1db WatchSource:0}: Error finding container 777fc22726bc541f42bbc32522bb51f19c8dca4d4bc8d882c5d56f1473f2e1db: Status 404 returned error can't find the container with id 777fc22726bc541f42bbc32522bb51f19c8dca4d4bc8d882c5d56f1473f2e1db Jul 31 22:58:26.935624 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:26.935588 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:27.016729 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.016644 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5"] Jul 31 22:58:27.019243 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:27.019207 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f6936db_6831_43bf_a6e4_fd33552c2657.slice/crio-bf33db0f8260e4f91516df81654ed07471c3166022618f46dda5fba5efde19d6 WatchSource:0}: Error finding container bf33db0f8260e4f91516df81654ed07471c3166022618f46dda5fba5efde19d6: Status 404 returned error can't find the container with id bf33db0f8260e4f91516df81654ed07471c3166022618f46dda5fba5efde19d6 Jul 31 22:58:27.022573 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.022549 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:27.236195 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.236153 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0689778d-1ccf-4e60-a5f0-72ad871a40ba-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:27.238630 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.238604 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0689778d-1ccf-4e60-a5f0-72ad871a40ba-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-5fcmv\" (UID: \"0689778d-1ccf-4e60-a5f0-72ad871a40ba\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:27.306467 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.306439 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:27.400769 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.400728 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:27.401379 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.401336 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 22:58:27.403771 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.403750 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.410983 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.410239 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 31 22:58:27.410983 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.410270 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rdj2n\"" Jul 31 22:58:27.410983 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.410326 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 31 22:58:27.410983 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.410447 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 31 22:58:27.410983 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.410529 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 31 22:58:27.410983 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.410562 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 31 22:58:27.410983 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.410779 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 31 22:58:27.410983 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.410799 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 31 22:58:27.410983 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.410849 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 31 22:58:27.410983 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.410782 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 31 22:58:27.437334 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.437296 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/74863722-c1da-4905-93e4-8f0ecfb66d64-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.437488 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.437404 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-config-volume\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.437488 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.437447 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-web-config\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.437488 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.437485 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74863722-c1da-4905-93e4-8f0ecfb66d64-config-out\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.437658 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.437522 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9pzp\" (UniqueName: \"kubernetes.io/projected/74863722-c1da-4905-93e4-8f0ecfb66d64-kube-api-access-w9pzp\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.437658 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.437544 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.437658 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.437582 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.437783 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.437656 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74863722-c1da-4905-93e4-8f0ecfb66d64-tls-assets\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.437783 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.437685 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.437783 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.437711 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74863722-c1da-4905-93e4-8f0ecfb66d64-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.437921 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.437799 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.437921 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.437854 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.437921 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.437879 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74863722-c1da-4905-93e4-8f0ecfb66d64-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.444120 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.444087 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 22:58:27.486036 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.485994 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" Jul 31 22:58:27.520834 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.520799 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" event={"ID":"0f6936db-6831-43bf-a6e4-fd33552c2657","Type":"ContainerStarted","Data":"06ac8870e5fcc07a9eb2fb63616148eb6c12c08fe3ab0bf3780fd7e7a22565de"} Jul 31 22:58:27.520999 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.520873 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" event={"ID":"0f6936db-6831-43bf-a6e4-fd33552c2657","Type":"ContainerStarted","Data":"1430eece51c6e543d25c2fd0b39ef76a81d6f9f5fa9adab1a358c958ece9a1ae"} Jul 31 22:58:27.520999 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.520904 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" event={"ID":"0f6936db-6831-43bf-a6e4-fd33552c2657","Type":"ContainerStarted","Data":"bf33db0f8260e4f91516df81654ed07471c3166022618f46dda5fba5efde19d6"} Jul 31 22:58:27.522335 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.522306 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pj2bq" event={"ID":"189420ca-6274-47bb-b106-399b9002d242","Type":"ContainerStarted","Data":"777fc22726bc541f42bbc32522bb51f19c8dca4d4bc8d882c5d56f1473f2e1db"} Jul 31 22:58:27.540637 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.540540 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74863722-c1da-4905-93e4-8f0ecfb66d64-tls-assets\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.540637 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.540583 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.540637 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.540613 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74863722-c1da-4905-93e4-8f0ecfb66d64-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.540637 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.540641 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.541181 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.540659 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.541181 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.540676 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74863722-c1da-4905-93e4-8f0ecfb66d64-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.541181 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.540705 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/74863722-c1da-4905-93e4-8f0ecfb66d64-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.541181 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.540756 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-config-volume\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.541181 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.540789 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-web-config\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.541181 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.540822 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74863722-c1da-4905-93e4-8f0ecfb66d64-config-out\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.541181 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.540844 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9pzp\" (UniqueName: \"kubernetes.io/projected/74863722-c1da-4905-93e4-8f0ecfb66d64-kube-api-access-w9pzp\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.541181 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.540872 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.541181 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.540922 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.542217 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:58:27.541654 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/74863722-c1da-4905-93e4-8f0ecfb66d64-alertmanager-trusted-ca-bundle podName:74863722-c1da-4905-93e4-8f0ecfb66d64 nodeName:}" failed. No retries permitted until 2026-07-31 22:58:28.041627149 +0000 UTC m=+64.514071333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/74863722-c1da-4905-93e4-8f0ecfb66d64-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "74863722-c1da-4905-93e4-8f0ecfb66d64") : configmap references non-existent config key: ca-bundle.crt Jul 31 22:58:27.542217 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.541946 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/74863722-c1da-4905-93e4-8f0ecfb66d64-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.542217 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.542195 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74863722-c1da-4905-93e4-8f0ecfb66d64-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.544672 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.544314 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74863722-c1da-4905-93e4-8f0ecfb66d64-config-out\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.546657 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.546541 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74863722-c1da-4905-93e4-8f0ecfb66d64-tls-assets\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.546907 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.546882 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.547336 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.547310 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.547930 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.547902 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-config-volume\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.548750 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.548532 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.554527 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.554474 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-web-config\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.555334 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.555301 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.555445 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.555343 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.558542 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.558433 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9pzp\" (UniqueName: \"kubernetes.io/projected/74863722-c1da-4905-93e4-8f0ecfb66d64-kube-api-access-w9pzp\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:27.627680 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.627624 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:27.681453 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:27.681404 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv"] Jul 31 22:58:27.684463 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:27.684425 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0689778d_1ccf_4e60_a5f0_72ad871a40ba.slice/crio-1d0813aa709cdf41081f9e66ea223d56d1a5603b743bc404583a095deaa1c526 WatchSource:0}: Error finding container 1d0813aa709cdf41081f9e66ea223d56d1a5603b743bc404583a095deaa1c526: Status 404 returned error can't find the container with id 1d0813aa709cdf41081f9e66ea223d56d1a5603b743bc404583a095deaa1c526 Jul 31 22:58:28.045405 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.045321 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74863722-c1da-4905-93e4-8f0ecfb66d64-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:28.046404 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.046373 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74863722-c1da-4905-93e4-8f0ecfb66d64-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:28.315306 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.315267 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:58:28.491560 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.491526 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 22:58:28.493730 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:28.493699 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74863722_c1da_4905_93e4_8f0ecfb66d64.slice/crio-7f3c8618c6dcee7e6fa0218f741871782578599e44c18cf91fac237af6e8d120 WatchSource:0}: Error finding container 7f3c8618c6dcee7e6fa0218f741871782578599e44c18cf91fac237af6e8d120: Status 404 returned error can't find the container with id 7f3c8618c6dcee7e6fa0218f741871782578599e44c18cf91fac237af6e8d120 Jul 31 22:58:28.507061 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.507031 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8"] Jul 31 22:58:28.510320 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.509833 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.513216 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.513191 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 31 22:58:28.513559 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.513402 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 31 22:58:28.513559 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.513432 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 31 22:58:28.513559 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.513552 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 31 22:58:28.513847 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.513807 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 31 22:58:28.513999 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.513977 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7ru17eekp1gi9\"" Jul 31 22:58:28.514130 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.514114 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-78f9t\"" Jul 31 22:58:28.526808 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.526774 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8"] Jul 31 22:58:28.529293 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.529265 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" event={"ID":"0f6936db-6831-43bf-a6e4-fd33552c2657","Type":"ContainerStarted","Data":"97d3d9a75ae745611d393250e82e2f801dcdd977b9d8ee0d8a18584d35e41719"} Jul 31 22:58:28.530864 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.530817 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74863722-c1da-4905-93e4-8f0ecfb66d64","Type":"ContainerStarted","Data":"7f3c8618c6dcee7e6fa0218f741871782578599e44c18cf91fac237af6e8d120"} Jul 31 22:58:28.532690 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.532663 2561 generic.go:356] "Generic (PLEG): container finished" podID="189420ca-6274-47bb-b106-399b9002d242" containerID="351f81828ba3672ef481b840baf6b383744e93fccd76af8d03032309beef23ac" exitCode=0 Jul 31 22:58:28.532813 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.532748 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pj2bq" event={"ID":"189420ca-6274-47bb-b106-399b9002d242","Type":"ContainerDied","Data":"351f81828ba3672ef481b840baf6b383744e93fccd76af8d03032309beef23ac"} Jul 31 22:58:28.534133 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.534102 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" event={"ID":"0689778d-1ccf-4e60-a5f0-72ad871a40ba","Type":"ContainerStarted","Data":"1d0813aa709cdf41081f9e66ea223d56d1a5603b743bc404583a095deaa1c526"} Jul 31 22:58:28.549776 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.549691 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trj5f\" (UniqueName: \"kubernetes.io/projected/3744bbd5-b118-4aeb-a4d1-0453f53b6256-kube-api-access-trj5f\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.549776 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.549772 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.549961 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.549809 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-thanos-querier-tls\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.549961 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.549893 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.550063 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.549962 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.550063 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.549995 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3744bbd5-b118-4aeb-a4d1-0453f53b6256-metrics-client-ca\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.550063 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.550020 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-grpc-tls\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.550235 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.550200 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.584604 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.584548 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdfg5" podStartSLOduration=1.569198043 podStartE2EDuration="2.584520819s" totalImagesPullingTime="1.015322776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:28.582358444 +0000 UTC m=+65.054802628" watchObservedRunningTime="2026-07-31 22:58:28.584520819 +0000 UTC m=+65.056964981" Jul 31 22:58:28.652022 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.651972 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.652165 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.652073 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trj5f\" (UniqueName: \"kubernetes.io/projected/3744bbd5-b118-4aeb-a4d1-0453f53b6256-kube-api-access-trj5f\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.652165 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.652114 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.652165 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.652148 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-thanos-querier-tls\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.652383 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.652349 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.652528 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.652421 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.652528 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.652459 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3744bbd5-b118-4aeb-a4d1-0453f53b6256-metrics-client-ca\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.652528 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.652488 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-grpc-tls\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.653259 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.653197 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3744bbd5-b118-4aeb-a4d1-0453f53b6256-metrics-client-ca\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.655201 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.655146 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-thanos-querier-tls\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.655322 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.655299 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.655590 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.655556 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.655781 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.655753 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.656005 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.655951 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.656178 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.656159 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3744bbd5-b118-4aeb-a4d1-0453f53b6256-secret-grpc-tls\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.667483 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.667456 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trj5f\" (UniqueName: \"kubernetes.io/projected/3744bbd5-b118-4aeb-a4d1-0453f53b6256-kube-api-access-trj5f\") pod \"thanos-querier-5fb6f66bf9-9qsd8\" (UID: \"3744bbd5-b118-4aeb-a4d1-0453f53b6256\") " pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:28.821883 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:28.821793 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:29.129260 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.129226 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8"] Jul 31 22:58:29.138016 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:29.137981 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3744bbd5_b118_4aeb_a4d1_0453f53b6256.slice/crio-e1f4206d6ddb7fa158e175527ae05b44ba8ffc33bbc9d03b005ebf51488c0cac WatchSource:0}: Error finding container e1f4206d6ddb7fa158e175527ae05b44ba8ffc33bbc9d03b005ebf51488c0cac: Status 404 returned error can't find the container with id e1f4206d6ddb7fa158e175527ae05b44ba8ffc33bbc9d03b005ebf51488c0cac Jul 31 22:58:29.376275 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.376182 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:29.480788 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.480753 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:29.487492 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.487464 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zc76w" Jul 31 22:58:29.542416 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.541633 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pj2bq" event={"ID":"189420ca-6274-47bb-b106-399b9002d242","Type":"ContainerStarted","Data":"7d9387458195729e403752ccff396d3a0ecbd722ad983283fb3c9cc88c03a544"} Jul 31 22:58:29.542416 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.541702 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pj2bq" event={"ID":"189420ca-6274-47bb-b106-399b9002d242","Type":"ContainerStarted","Data":"e211b1d68742750465a0ef0e98572621cf01a989501795ba70ec7cea18653c48"} Jul 31 22:58:29.545092 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.545060 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" event={"ID":"0689778d-1ccf-4e60-a5f0-72ad871a40ba","Type":"ContainerStarted","Data":"08b3a7d52719ac8b29bd53db6c3091a7f7c472964e94f283ba9d0c331b993d5f"} Jul 31 22:58:29.545233 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.545116 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" event={"ID":"0689778d-1ccf-4e60-a5f0-72ad871a40ba","Type":"ContainerStarted","Data":"b04ad04e083015407bb3edf0c437496c0aacf5678ffd0a343faa86b58f42a80f"} Jul 31 22:58:29.545233 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.545139 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" event={"ID":"0689778d-1ccf-4e60-a5f0-72ad871a40ba","Type":"ContainerStarted","Data":"fd7bc2457d503bdd5b1a6821553a9a5d0e20e50510a81f4f6cd5286eea654d72"} Jul 31 22:58:29.547417 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.547381 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" event={"ID":"3744bbd5-b118-4aeb-a4d1-0453f53b6256","Type":"ContainerStarted","Data":"e1f4206d6ddb7fa158e175527ae05b44ba8ffc33bbc9d03b005ebf51488c0cac"} Jul 31 22:58:29.565148 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.565082 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pj2bq" podStartSLOduration=2.276454168 podStartE2EDuration="3.565031184s" totalImagesPullingTime="1.219352957s" totalInitContainerRuntime="69.224059ms" isStatefulPod=true podCreationTimestamp="2026-07-31 22:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:29.563864357 +0000 UTC m=+66.036308565" watchObservedRunningTime="2026-07-31 22:58:29.565031184 +0000 UTC m=+66.037475368" Jul 31 22:58:29.582383 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.582346 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:29.596840 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.596780 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5fcmv" podStartSLOduration=2.274524442 podStartE2EDuration="3.5967573s" totalImagesPullingTime="1.322232858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:29.595735496 +0000 UTC m=+66.068179680" watchObservedRunningTime="2026-07-31 22:58:29.5967573 +0000 UTC m=+66.069201483" Jul 31 22:58:29.702610 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.702527 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:29.864146 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.864113 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:58:29.864319 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.864229 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs\") pod \"network-metrics-daemon-44zk2\" (UID: \"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e\") " pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:58:29.868019 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.867986 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 31 22:58:29.868388 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.868369 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 31 22:58:29.877174 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.877150 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40c30d12-8f96-4d63-a17e-5a3c101f2439-original-pull-secret\") pod \"global-pull-secret-syncer-lqgnq\" (UID: \"40c30d12-8f96-4d63-a17e-5a3c101f2439\") " pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:58:29.877271 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.877205 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e-metrics-certs\") pod \"network-metrics-daemon-44zk2\" (UID: \"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e\") " pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:58:29.883562 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.883532 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:29.964914 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.964833 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zbrk\" (UniqueName: \"kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk\") pod \"network-check-target-9g22p\" (UID: \"cd1248a5-5ddf-4647-b417-5099076f0133\") " pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:58:29.967653 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.967625 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 31 22:58:29.981167 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.981138 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 31 22:58:29.988232 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:29.988203 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zbrk\" (UniqueName: \"kubernetes.io/projected/cd1248a5-5ddf-4647-b417-5099076f0133-kube-api-access-2zbrk\") pod \"network-check-target-9g22p\" (UID: \"cd1248a5-5ddf-4647-b417-5099076f0133\") " pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:58:30.139550 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:30.139519 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h8t45\"" Jul 31 22:58:30.141628 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:30.141602 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lqgnq" Jul 31 22:58:30.146655 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:30.146631 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44zk2" Jul 31 22:58:30.148184 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:30.148167 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jtx5g\"" Jul 31 22:58:30.155901 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:30.155879 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:58:30.310557 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:30.310260 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-44zk2"] Jul 31 22:58:30.313709 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:30.313678 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58daa0fe_d3d1_4c52_8d5e_94bc5c3f5b3e.slice/crio-439e16c616963d802a39a46293802897a99df1451b94334b6b4ff3b6f64b528f WatchSource:0}: Error finding container 439e16c616963d802a39a46293802897a99df1451b94334b6b4ff3b6f64b528f: Status 404 returned error can't find the container with id 439e16c616963d802a39a46293802897a99df1451b94334b6b4ff3b6f64b528f Jul 31 22:58:30.332387 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:30.332358 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lqgnq"] Jul 31 22:58:30.335352 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:30.335321 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c30d12_8f96_4d63_a17e_5a3c101f2439.slice/crio-8b5b99c4e49d2939136516aebcc966a54386d01f0b87cd5cc6b4ef352d06d32f WatchSource:0}: Error finding container 8b5b99c4e49d2939136516aebcc966a54386d01f0b87cd5cc6b4ef352d06d32f: Status 404 returned error can't find the container with id 8b5b99c4e49d2939136516aebcc966a54386d01f0b87cd5cc6b4ef352d06d32f Jul 31 22:58:30.348898 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:30.348861 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9g22p"] Jul 31 22:58:30.352581 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:30.352552 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd1248a5_5ddf_4647_b417_5099076f0133.slice/crio-fc8c3605b0f197c277544cbefe6bc9be99a15e54f719d28bd5033d5fd3975171 WatchSource:0}: Error finding container fc8c3605b0f197c277544cbefe6bc9be99a15e54f719d28bd5033d5fd3975171: Status 404 returned error can't find the container with id fc8c3605b0f197c277544cbefe6bc9be99a15e54f719d28bd5033d5fd3975171 Jul 31 22:58:30.554768 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:30.554732 2561 generic.go:356] "Generic (PLEG): container finished" podID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerID="e7a3d60f189c5d304c5a51123b7535414b71b0cbc97b07a2b05060ffa9785c4b" exitCode=0 Jul 31 22:58:30.555225 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:30.554834 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74863722-c1da-4905-93e4-8f0ecfb66d64","Type":"ContainerDied","Data":"e7a3d60f189c5d304c5a51123b7535414b71b0cbc97b07a2b05060ffa9785c4b"} Jul 31 22:58:30.557058 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:30.557034 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-44zk2" event={"ID":"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e","Type":"ContainerStarted","Data":"439e16c616963d802a39a46293802897a99df1451b94334b6b4ff3b6f64b528f"} Jul 31 22:58:30.558345 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:30.558316 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9g22p" event={"ID":"cd1248a5-5ddf-4647-b417-5099076f0133","Type":"ContainerStarted","Data":"fc8c3605b0f197c277544cbefe6bc9be99a15e54f719d28bd5033d5fd3975171"} Jul 31 22:58:30.559654 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:30.559567 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lqgnq" event={"ID":"40c30d12-8f96-4d63-a17e-5a3c101f2439","Type":"ContainerStarted","Data":"8b5b99c4e49d2939136516aebcc966a54386d01f0b87cd5cc6b4ef352d06d32f"} Jul 31 22:58:31.009585 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.009238 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5c7f464cdb-t2x7g"] Jul 31 22:58:31.014480 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.013552 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.018411 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.018012 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 31 22:58:31.018411 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.018246 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 31 22:58:31.022539 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.020799 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-zl68v\"" Jul 31 22:58:31.022539 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.020790 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-e6kksmtoblgqg\"" Jul 31 22:58:31.022539 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.021065 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 31 22:58:31.022539 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.021186 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 31 22:58:31.033188 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.033116 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c7f464cdb-t2x7g"] Jul 31 22:58:31.075443 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.075204 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4ed3765c-6eff-4f01-a894-52b84f544755-secret-metrics-server-client-certs\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.075443 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.075269 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4ed3765c-6eff-4f01-a894-52b84f544755-audit-log\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.075443 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.075314 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ed3765c-6eff-4f01-a894-52b84f544755-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.075443 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.075344 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ed3765c-6eff-4f01-a894-52b84f544755-client-ca-bundle\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.075443 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.075382 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4ed3765c-6eff-4f01-a894-52b84f544755-secret-metrics-server-tls\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.075979 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.075661 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rlbq\" (UniqueName: \"kubernetes.io/projected/4ed3765c-6eff-4f01-a894-52b84f544755-kube-api-access-2rlbq\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.075979 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.075785 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4ed3765c-6eff-4f01-a894-52b84f544755-metrics-server-audit-profiles\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.109750 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.109363 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j"] Jul 31 22:58:31.112130 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.112051 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:31.113834 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.113793 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j" Jul 31 22:58:31.117892 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.117576 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 31 22:58:31.117892 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.117654 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-dhbnj\"" Jul 31 22:58:31.120852 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.120822 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j"] Jul 31 22:58:31.177236 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.177198 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4ed3765c-6eff-4f01-a894-52b84f544755-secret-metrics-server-tls\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.177418 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.177249 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rlbq\" (UniqueName: \"kubernetes.io/projected/4ed3765c-6eff-4f01-a894-52b84f544755-kube-api-access-2rlbq\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.177418 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.177320 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4ed3765c-6eff-4f01-a894-52b84f544755-metrics-server-audit-profiles\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.177418 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.177365 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzq5t\" (UniqueName: \"kubernetes.io/projected/a16f9d70-421d-414d-aca7-f2ea06d89930-kube-api-access-fzq5t\") pod \"monitoring-plugin-6dffbbd67-4z28j\" (UID: \"a16f9d70-421d-414d-aca7-f2ea06d89930\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j" Jul 31 22:58:31.177418 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.177399 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4ed3765c-6eff-4f01-a894-52b84f544755-secret-metrics-server-client-certs\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.177749 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.177426 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a16f9d70-421d-414d-aca7-f2ea06d89930-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-4z28j\" (UID: \"a16f9d70-421d-414d-aca7-f2ea06d89930\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j" Jul 31 22:58:31.177749 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.177456 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4ed3765c-6eff-4f01-a894-52b84f544755-audit-log\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.177749 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.177495 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ed3765c-6eff-4f01-a894-52b84f544755-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.177749 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.177544 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ed3765c-6eff-4f01-a894-52b84f544755-client-ca-bundle\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.178615 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.178576 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4ed3765c-6eff-4f01-a894-52b84f544755-metrics-server-audit-profiles\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.179486 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.179462 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4ed3765c-6eff-4f01-a894-52b84f544755-audit-log\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.179604 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.179493 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ed3765c-6eff-4f01-a894-52b84f544755-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.181901 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.180400 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4ed3765c-6eff-4f01-a894-52b84f544755-secret-metrics-server-tls\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.183943 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.182134 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ed3765c-6eff-4f01-a894-52b84f544755-client-ca-bundle\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.183943 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.182621 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4ed3765c-6eff-4f01-a894-52b84f544755-secret-metrics-server-client-certs\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.190285 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.190231 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rlbq\" (UniqueName: \"kubernetes.io/projected/4ed3765c-6eff-4f01-a894-52b84f544755-kube-api-access-2rlbq\") pod \"metrics-server-5c7f464cdb-t2x7g\" (UID: \"4ed3765c-6eff-4f01-a894-52b84f544755\") " pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.279776 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.278815 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzq5t\" (UniqueName: \"kubernetes.io/projected/a16f9d70-421d-414d-aca7-f2ea06d89930-kube-api-access-fzq5t\") pod \"monitoring-plugin-6dffbbd67-4z28j\" (UID: \"a16f9d70-421d-414d-aca7-f2ea06d89930\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j" Jul 31 22:58:31.279776 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.278876 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a16f9d70-421d-414d-aca7-f2ea06d89930-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-4z28j\" (UID: \"a16f9d70-421d-414d-aca7-f2ea06d89930\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j" Jul 31 22:58:31.282841 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.282784 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a16f9d70-421d-414d-aca7-f2ea06d89930-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-4z28j\" (UID: \"a16f9d70-421d-414d-aca7-f2ea06d89930\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j" Jul 31 22:58:31.291965 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.291904 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzq5t\" (UniqueName: \"kubernetes.io/projected/a16f9d70-421d-414d-aca7-f2ea06d89930-kube-api-access-fzq5t\") pod \"monitoring-plugin-6dffbbd67-4z28j\" (UID: \"a16f9d70-421d-414d-aca7-f2ea06d89930\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j" Jul 31 22:58:31.318964 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.318925 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:31.328613 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.328160 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:31.427931 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.427895 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j" Jul 31 22:58:31.628657 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.628629 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c7f464cdb-t2x7g"] Jul 31 22:58:31.637079 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:31.637039 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ed3765c_6eff_4f01_a894_52b84f544755.slice/crio-5446a4a9681465d2ec407d385a63997af5ba6c14bdcaefd8aa7362527e51d161 WatchSource:0}: Error finding container 5446a4a9681465d2ec407d385a63997af5ba6c14bdcaefd8aa7362527e51d161: Status 404 returned error can't find the container with id 5446a4a9681465d2ec407d385a63997af5ba6c14bdcaefd8aa7362527e51d161 Jul 31 22:58:31.652745 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:31.651382 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j"] Jul 31 22:58:31.659257 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:31.659218 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda16f9d70_421d_414d_aca7_f2ea06d89930.slice/crio-11ce650a1c991d04e4713584b57646d78bdc047a900df418e04109b6ebeb20eb WatchSource:0}: Error finding container 11ce650a1c991d04e4713584b57646d78bdc047a900df418e04109b6ebeb20eb: Status 404 returned error can't find the container with id 11ce650a1c991d04e4713584b57646d78bdc047a900df418e04109b6ebeb20eb Jul 31 22:58:32.576313 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.576238 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" event={"ID":"3744bbd5-b118-4aeb-a4d1-0453f53b6256","Type":"ContainerStarted","Data":"52f537ec40017b4128ef1c57f0c8f7d4687dce7bfb7833ada06050949255b723"} Jul 31 22:58:32.576486 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.576324 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" event={"ID":"3744bbd5-b118-4aeb-a4d1-0453f53b6256","Type":"ContainerStarted","Data":"b02553d52d0a244bc32d0bef3ef7eafe2265548515e36cd91e483e611c85ab65"} Jul 31 22:58:32.576486 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.576366 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" event={"ID":"3744bbd5-b118-4aeb-a4d1-0453f53b6256","Type":"ContainerStarted","Data":"f3e3406f0fce7244ac13a1fac0dba1b4ff6f3a9ecb13d838bd18d6df962bdbdb"} Jul 31 22:58:32.584273 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.584238 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j" event={"ID":"a16f9d70-421d-414d-aca7-f2ea06d89930","Type":"ContainerStarted","Data":"11ce650a1c991d04e4713584b57646d78bdc047a900df418e04109b6ebeb20eb"} Jul 31 22:58:32.586213 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.586183 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" event={"ID":"4ed3765c-6eff-4f01-a894-52b84f544755","Type":"ContainerStarted","Data":"5446a4a9681465d2ec407d385a63997af5ba6c14bdcaefd8aa7362527e51d161"} Jul 31 22:58:32.773793 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.773671 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:32.785025 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.783019 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 31 22:58:32.805530 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.805176 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.812987 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.812270 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 31 22:58:32.812987 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.812566 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 31 22:58:32.812987 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.812889 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 31 22:58:32.813417 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.813401 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 31 22:58:32.814045 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.814028 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 31 22:58:32.815046 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.815030 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 31 22:58:32.815775 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.815761 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 31 22:58:32.816198 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.816184 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 31 22:58:32.838925 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.816367 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 31 22:58:32.838925 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.817048 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-78r9ui4g8hkem\"" Jul 31 22:58:32.838925 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.817828 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 31 22:58:32.842187 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.839880 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nzcdr\"" Jul 31 22:58:32.842187 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.840101 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 31 22:58:32.842187 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.840271 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 31 22:58:32.842187 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.840573 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 31 22:58:32.842187 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.841953 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 31 22:58:32.844058 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.843824 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 31 22:58:32.896065 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.895875 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-config-out\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896065 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.895920 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896065 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.895948 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896065 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.895970 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896065 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896000 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-web-config\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896065 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896020 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896065 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896044 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2mwm\" (UniqueName: \"kubernetes.io/projected/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-kube-api-access-c2mwm\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896065 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896073 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896588 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896148 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896588 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896204 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896588 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896239 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896588 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896313 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896588 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896381 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896588 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896416 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896588 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896528 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896588 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896556 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.896588 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896584 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.897054 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896606 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.897054 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896632 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.897054 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.896659 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-config\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.904181 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.904150 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:32.998890 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.997900 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.998890 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.997947 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.998890 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.997975 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.998890 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.998005 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.998890 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.998032 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.998890 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.998106 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.998890 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.998130 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.998890 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.998157 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.998890 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.998179 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.998890 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.998205 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.998890 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.998233 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-config\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.998890 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.998282 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-config-out\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.998890 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.998305 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.998890 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.998332 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.998890 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.998357 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:32.999752 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.999526 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.000037 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.998392 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-web-config\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.000037 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.999903 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.000037 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.999932 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2mwm\" (UniqueName: \"kubernetes.io/projected/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-kube-api-access-c2mwm\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.000037 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.999966 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.000037 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.999994 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.000724 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:32.999995 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.000810 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.000747 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.001091 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.001067 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.002181 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.001697 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.004147 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.002495 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.004147 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.003065 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.004147 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.003678 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.004828 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.004802 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.006812 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.006781 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:33.006907 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.006862 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.007345 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.007323 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.007540 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.007494 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.010435 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.010392 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.012901 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.012667 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.012901 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.012862 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-config-out\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.014065 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.013959 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.014540 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.014475 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-web-config\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.014644 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.014623 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-config\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.014919 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.014729 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.017447 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.017368 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2mwm\" (UniqueName: \"kubernetes.io/projected/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-kube-api-access-c2mwm\") pod \"prometheus-k8s-0\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.147085 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.147001 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:33.211067 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.210966 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:33.278101 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:33.278037 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:35.545677 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:35.545637 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:35.616267 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:35.616190 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:36.939756 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:36.939661 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 31 22:58:36.947642 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:58:36.946041 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea6c3200_1f3d_437b_9d52_26c4c3b5a4a4.slice/crio-83ee144bd652e59f1ba6c855b1cfd2dec971ebaff46eb55d31062811a27b1977 WatchSource:0}: Error finding container 83ee144bd652e59f1ba6c855b1cfd2dec971ebaff46eb55d31062811a27b1977: Status 404 returned error can't find the container with id 83ee144bd652e59f1ba6c855b1cfd2dec971ebaff46eb55d31062811a27b1977 Jul 31 22:58:37.610143 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.610104 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9g22p" event={"ID":"cd1248a5-5ddf-4647-b417-5099076f0133","Type":"ContainerStarted","Data":"675b1e7a5eacdfcb6196c368f036512377ec8f4e5d508e2ddbf446521e818ee5"} Jul 31 22:58:37.610343 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.610227 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:58:37.611775 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.611741 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lqgnq" event={"ID":"40c30d12-8f96-4d63-a17e-5a3c101f2439","Type":"ContainerStarted","Data":"7e3024e7db0af48c1e8a5263ab4a311d539eea79e61b748e850035fd0b8eb6f9"} Jul 31 22:58:37.613224 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.613202 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j" event={"ID":"a16f9d70-421d-414d-aca7-f2ea06d89930","Type":"ContainerStarted","Data":"52b8debd55c76c78032af595d498416a4b6431dc813950f0f1ca4e1d16727769"} Jul 31 22:58:37.613373 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.613354 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j" Jul 31 22:58:37.617052 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.617030 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74863722-c1da-4905-93e4-8f0ecfb66d64","Type":"ContainerStarted","Data":"0d6db748d198ffe2c3ce5c46bc95a3442e3ba02203d1e5babe4099f8570fd534"} Jul 31 22:58:37.617192 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.617093 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74863722-c1da-4905-93e4-8f0ecfb66d64","Type":"ContainerStarted","Data":"e911f5437ee76a1002e5f4479e62611fa77c914635f2d86bebf629ccee8cf02f"} Jul 31 22:58:37.617192 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.617135 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74863722-c1da-4905-93e4-8f0ecfb66d64","Type":"ContainerStarted","Data":"174cc57f7c58e589593d0d605b122247c05992146fc56044db5f4f891d085eec"} Jul 31 22:58:37.617192 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.617183 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74863722-c1da-4905-93e4-8f0ecfb66d64","Type":"ContainerStarted","Data":"eed2a6a5ba9f219ee41455fdc478c36060b6b6cb011cffebdf84a1e2ed547c41"} Jul 31 22:58:37.617351 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.617222 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74863722-c1da-4905-93e4-8f0ecfb66d64","Type":"ContainerStarted","Data":"2f7e592b0f0a89b88af44775b4b59656852deb90300e1dd7e6727ca2f0647271"} Jul 31 22:58:37.617351 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.617265 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74863722-c1da-4905-93e4-8f0ecfb66d64","Type":"ContainerStarted","Data":"7d4c72bc6568ac8c17956d28939d96ea45c6c7e0e097171c703770cad3d72e0c"} Jul 31 22:58:37.619077 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.619053 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-44zk2" event={"ID":"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e","Type":"ContainerStarted","Data":"1fb2998c2dcdf92e1e0eeab90c2fcb4a3baac3ae89171c8727a82b32dea9156e"} Jul 31 22:58:37.619345 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.619103 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-44zk2" event={"ID":"58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e","Type":"ContainerStarted","Data":"d7f1d8c309346bfd2a116cdcf62e445141f427a98e3437952b05d865b0dea797"} Jul 31 22:58:37.619697 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.619676 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j" Jul 31 22:58:37.620741 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.620721 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" event={"ID":"4ed3765c-6eff-4f01-a894-52b84f544755","Type":"ContainerStarted","Data":"72a59715186930558ac9c12eefc657085b15a2ec7c8aeab7646f0cd8e6d1345b"} Jul 31 22:58:37.622178 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.622157 2561 generic.go:356] "Generic (PLEG): container finished" podID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerID="429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa" exitCode=0 Jul 31 22:58:37.622277 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.622219 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4","Type":"ContainerDied","Data":"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa"} Jul 31 22:58:37.622328 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.622287 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4","Type":"ContainerStarted","Data":"83ee144bd652e59f1ba6c855b1cfd2dec971ebaff46eb55d31062811a27b1977"} Jul 31 22:58:37.625562 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.625539 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" event={"ID":"3744bbd5-b118-4aeb-a4d1-0453f53b6256","Type":"ContainerStarted","Data":"424fad0e640383879a89d2a7840fdbc71ba3444c0c10e1d9bc864dde4c18f7db"} Jul 31 22:58:37.625648 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.625601 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" event={"ID":"3744bbd5-b118-4aeb-a4d1-0453f53b6256","Type":"ContainerStarted","Data":"d4477d77121327adfde67e72a75dcf445e0e511d4e957bb4b73e100bf5381b2f"} Jul 31 22:58:37.625648 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.625637 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" event={"ID":"3744bbd5-b118-4aeb-a4d1-0453f53b6256","Type":"ContainerStarted","Data":"9fea82e59c3b459d88b1e82c7cfaec1730e6523e6e01dbf8445edacf6612bdb7"} Jul 31 22:58:37.625778 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.625735 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:37.630808 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.630753 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9g22p" podStartSLOduration=67.184229463 podStartE2EDuration="1m13.630734949s" totalImagesPullingTime="6.446505486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:37.629056377 +0000 UTC m=+74.101500558" watchObservedRunningTime="2026-07-31 22:58:37.630734949 +0000 UTC m=+74.103179135" Jul 31 22:58:37.633739 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.633721 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" Jul 31 22:58:37.658872 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.658792 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.192283552 podStartE2EDuration="10.65877201s" totalImagesPullingTime="7.453307702s" totalInitContainerRuntime="13.180756ms" isStatefulPod=false podCreationTimestamp="2026-07-31 22:58:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:37.658007056 +0000 UTC m=+74.130451240" watchObservedRunningTime="2026-07-31 22:58:37.65877201 +0000 UTC m=+74.131216193" Jul 31 22:58:37.708061 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.708012 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lqgnq" podStartSLOduration=67.224701944 podStartE2EDuration="1m13.707994631s" totalImagesPullingTime="6.483292687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:37.707008049 +0000 UTC m=+74.179452231" watchObservedRunningTime="2026-07-31 22:58:37.707994631 +0000 UTC m=+74.180438813" Jul 31 22:58:37.708897 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.708846 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4z28j" podStartSLOduration=1.5995124120000002 podStartE2EDuration="6.708829372s" totalImagesPullingTime="5.10931696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:37.68008336 +0000 UTC m=+74.152527540" watchObservedRunningTime="2026-07-31 22:58:37.708829372 +0000 UTC m=+74.181273556" Jul 31 22:58:37.726162 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.726104 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-44zk2" podStartSLOduration=67.273261505 podStartE2EDuration="1m13.726091855s" totalImagesPullingTime="6.45283035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:37.724008556 +0000 UTC m=+74.196452742" watchObservedRunningTime="2026-07-31 22:58:37.726091855 +0000 UTC m=+74.198536039" Jul 31 22:58:37.749373 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.749327 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5fb6f66bf9-9qsd8" podStartSLOduration=2.433223365 podStartE2EDuration="9.749311001s" totalImagesPullingTime="7.316087636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:58:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:37.748305324 +0000 UTC m=+74.220749527" watchObservedRunningTime="2026-07-31 22:58:37.749311001 +0000 UTC m=+74.221755209" Jul 31 22:58:37.777157 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.777100 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" podStartSLOduration=2.617233831 podStartE2EDuration="7.77708094s" totalImagesPullingTime="5.159847109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:37.775404661 +0000 UTC m=+74.247848836" watchObservedRunningTime="2026-07-31 22:58:37.77708094 +0000 UTC m=+74.249525126" Jul 31 22:58:37.877142 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.877043 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:37.954110 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:37.954068 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:40.648881 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:40.648835 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:40.719205 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:40.719166 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:41.648815 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:41.648775 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4","Type":"ContainerStarted","Data":"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1"} Jul 31 22:58:41.648976 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:41.648897 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4","Type":"ContainerStarted","Data":"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976"} Jul 31 22:58:41.648976 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:41.648941 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4","Type":"ContainerStarted","Data":"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca"} Jul 31 22:58:41.649273 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:41.648980 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4","Type":"ContainerStarted","Data":"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32"} Jul 31 22:58:41.649273 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:41.649013 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4","Type":"ContainerStarted","Data":"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852"} Jul 31 22:58:41.649273 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:41.649047 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4","Type":"ContainerStarted","Data":"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04"} Jul 31 22:58:41.684725 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:41.684667 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.241511714 podStartE2EDuration="9.684646476s" totalImagesPullingTime="3.428072681s" totalInitContainerRuntime="15.062081ms" isStatefulPod=false podCreationTimestamp="2026-07-31 22:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:41.682979199 +0000 UTC m=+78.155423383" watchObservedRunningTime="2026-07-31 22:58:41.684646476 +0000 UTC m=+78.157090660" Jul 31 22:58:43.148210 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:43.148170 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:58:43.685756 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:43.685717 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:43.786836 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:43.786798 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:43.945163 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:43.945067 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:44.332649 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:44.332595 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:44.412568 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:44.412531 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:44.723777 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:44.723681 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:44.794658 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:44.794616 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:44.929108 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:44.929070 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:44.997337 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:44.997243 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:45.336594 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:45.336565 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:45.542633 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:45.542605 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:46.042965 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:46.042936 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:46.144014 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:46.143982 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:46.244271 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:46.244241 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:46.344886 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:46.344855 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:46.958559 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:46.958529 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:47.027069 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:47.027017 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:51.328786 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:51.328742 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:51.329258 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:51.328818 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:58:59.162847 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:59.162796 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:59.262385 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:58:59.262343 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:01.038193 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:01.038160 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:01.101958 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:01.101928 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:02.760303 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:02.760263 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:02.821070 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:02.821031 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:05.352076 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:05.352043 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:05.452753 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:05.452720 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:05.976924 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:05.976888 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:06.053306 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:06.053267 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:07.731467 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:07.731424 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:07.832478 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:07.832444 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:08.634125 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:08.634090 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9g22p" Jul 31 22:59:10.572067 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:10.572031 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:10.645247 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:10.645214 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:11.334603 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:11.334574 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:59:11.338586 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:11.338554 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5c7f464cdb-t2x7g" Jul 31 22:59:13.468264 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:13.468224 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:13.557386 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:13.557345 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:13.852813 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:13.852779 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:16.124027 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:16.123993 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:16.188702 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:16.188663 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:23.575811 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:23.575777 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:23.672931 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:23.672898 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:23.772313 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:23.772274 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:27.632055 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:27.632024 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:31.044819 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:31.044774 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:31.107643 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:31.107606 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:33.148055 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:33.148015 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:33.168722 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:33.168695 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:33.870454 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:33.870425 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:45.656280 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:45.656243 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:46.134019 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.133977 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:46.735388 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.735356 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 22:59:46.735843 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.735659 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="alertmanager" containerID="cri-o://7d4c72bc6568ac8c17956d28939d96ea45c6c7e0e097171c703770cad3d72e0c" gracePeriod=120 Jul 31 22:59:46.735843 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.735731 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="kube-rbac-proxy-metric" containerID="cri-o://e911f5437ee76a1002e5f4479e62611fa77c914635f2d86bebf629ccee8cf02f" gracePeriod=120 Jul 31 22:59:46.735843 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.735749 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="kube-rbac-proxy-web" containerID="cri-o://eed2a6a5ba9f219ee41455fdc478c36060b6b6cb011cffebdf84a1e2ed547c41" gracePeriod=120 Jul 31 22:59:46.735843 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.735787 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="prom-label-proxy" containerID="cri-o://0d6db748d198ffe2c3ce5c46bc95a3442e3ba02203d1e5babe4099f8570fd534" gracePeriod=120 Jul 31 22:59:46.735843 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.735810 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="kube-rbac-proxy" containerID="cri-o://174cc57f7c58e589593d0d605b122247c05992146fc56044db5f4f891d085eec" gracePeriod=120 Jul 31 22:59:46.736100 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.735882 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="config-reloader" containerID="cri-o://2f7e592b0f0a89b88af44775b4b59656852deb90300e1dd7e6727ca2f0647271" gracePeriod=120 Jul 31 22:59:46.907423 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.907390 2561 generic.go:356] "Generic (PLEG): container finished" podID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerID="0d6db748d198ffe2c3ce5c46bc95a3442e3ba02203d1e5babe4099f8570fd534" exitCode=0 Jul 31 22:59:46.907423 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.907413 2561 generic.go:356] "Generic (PLEG): container finished" podID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerID="e911f5437ee76a1002e5f4479e62611fa77c914635f2d86bebf629ccee8cf02f" exitCode=0 Jul 31 22:59:46.907423 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.907419 2561 generic.go:356] "Generic (PLEG): container finished" podID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerID="174cc57f7c58e589593d0d605b122247c05992146fc56044db5f4f891d085eec" exitCode=0 Jul 31 22:59:46.907423 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.907424 2561 generic.go:356] "Generic (PLEG): container finished" podID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerID="2f7e592b0f0a89b88af44775b4b59656852deb90300e1dd7e6727ca2f0647271" exitCode=0 Jul 31 22:59:46.907423 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.907430 2561 generic.go:356] "Generic (PLEG): container finished" podID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerID="7d4c72bc6568ac8c17956d28939d96ea45c6c7e0e097171c703770cad3d72e0c" exitCode=0 Jul 31 22:59:46.907806 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.907462 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74863722-c1da-4905-93e4-8f0ecfb66d64","Type":"ContainerDied","Data":"0d6db748d198ffe2c3ce5c46bc95a3442e3ba02203d1e5babe4099f8570fd534"} Jul 31 22:59:46.907806 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.907572 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74863722-c1da-4905-93e4-8f0ecfb66d64","Type":"ContainerDied","Data":"e911f5437ee76a1002e5f4479e62611fa77c914635f2d86bebf629ccee8cf02f"} Jul 31 22:59:46.907806 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.907614 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74863722-c1da-4905-93e4-8f0ecfb66d64","Type":"ContainerDied","Data":"174cc57f7c58e589593d0d605b122247c05992146fc56044db5f4f891d085eec"} Jul 31 22:59:46.907806 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.907649 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74863722-c1da-4905-93e4-8f0ecfb66d64","Type":"ContainerDied","Data":"2f7e592b0f0a89b88af44775b4b59656852deb90300e1dd7e6727ca2f0647271"} Jul 31 22:59:46.907806 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:46.907688 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74863722-c1da-4905-93e4-8f0ecfb66d64","Type":"ContainerDied","Data":"7d4c72bc6568ac8c17956d28939d96ea45c6c7e0e097171c703770cad3d72e0c"} Jul 31 22:59:47.916319 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:47.916289 2561 generic.go:356] "Generic (PLEG): container finished" podID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerID="eed2a6a5ba9f219ee41455fdc478c36060b6b6cb011cffebdf84a1e2ed547c41" exitCode=0 Jul 31 22:59:47.916661 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:47.916341 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74863722-c1da-4905-93e4-8f0ecfb66d64","Type":"ContainerDied","Data":"eed2a6a5ba9f219ee41455fdc478c36060b6b6cb011cffebdf84a1e2ed547c41"} Jul 31 22:59:47.979889 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:47.979863 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:48.165066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.163424 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-main-tls\") pod \"74863722-c1da-4905-93e4-8f0ecfb66d64\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " Jul 31 22:59:48.165066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.163466 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy\") pod \"74863722-c1da-4905-93e4-8f0ecfb66d64\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " Jul 31 22:59:48.165066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.163522 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy-metric\") pod \"74863722-c1da-4905-93e4-8f0ecfb66d64\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " Jul 31 22:59:48.165066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.163552 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-cluster-tls-config\") pod \"74863722-c1da-4905-93e4-8f0ecfb66d64\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " Jul 31 22:59:48.165066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.163582 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-web-config\") pod \"74863722-c1da-4905-93e4-8f0ecfb66d64\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " Jul 31 22:59:48.165066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.163622 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-config-volume\") pod \"74863722-c1da-4905-93e4-8f0ecfb66d64\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " Jul 31 22:59:48.165066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.163664 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74863722-c1da-4905-93e4-8f0ecfb66d64-alertmanager-trusted-ca-bundle\") pod \"74863722-c1da-4905-93e4-8f0ecfb66d64\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " Jul 31 22:59:48.165066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.163703 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/74863722-c1da-4905-93e4-8f0ecfb66d64-alertmanager-main-db\") pod \"74863722-c1da-4905-93e4-8f0ecfb66d64\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " Jul 31 22:59:48.165066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.163726 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74863722-c1da-4905-93e4-8f0ecfb66d64-config-out\") pod \"74863722-c1da-4905-93e4-8f0ecfb66d64\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " Jul 31 22:59:48.165066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.163748 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74863722-c1da-4905-93e4-8f0ecfb66d64-tls-assets\") pod \"74863722-c1da-4905-93e4-8f0ecfb66d64\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " Jul 31 22:59:48.165066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.163832 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9pzp\" (UniqueName: \"kubernetes.io/projected/74863722-c1da-4905-93e4-8f0ecfb66d64-kube-api-access-w9pzp\") pod \"74863722-c1da-4905-93e4-8f0ecfb66d64\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " Jul 31 22:59:48.165066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.163861 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy-web\") pod \"74863722-c1da-4905-93e4-8f0ecfb66d64\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " Jul 31 22:59:48.165066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.163906 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74863722-c1da-4905-93e4-8f0ecfb66d64-metrics-client-ca\") pod \"74863722-c1da-4905-93e4-8f0ecfb66d64\" (UID: \"74863722-c1da-4905-93e4-8f0ecfb66d64\") " Jul 31 22:59:48.165066 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.164759 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74863722-c1da-4905-93e4-8f0ecfb66d64-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "74863722-c1da-4905-93e4-8f0ecfb66d64" (UID: "74863722-c1da-4905-93e4-8f0ecfb66d64"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:59:48.165849 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.165368 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74863722-c1da-4905-93e4-8f0ecfb66d64-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "74863722-c1da-4905-93e4-8f0ecfb66d64" (UID: "74863722-c1da-4905-93e4-8f0ecfb66d64"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:59:48.173933 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.170051 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74863722-c1da-4905-93e4-8f0ecfb66d64-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "74863722-c1da-4905-93e4-8f0ecfb66d64" (UID: "74863722-c1da-4905-93e4-8f0ecfb66d64"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:59:48.173933 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.173878 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "74863722-c1da-4905-93e4-8f0ecfb66d64" (UID: "74863722-c1da-4905-93e4-8f0ecfb66d64"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:48.175003 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.174944 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "74863722-c1da-4905-93e4-8f0ecfb66d64" (UID: "74863722-c1da-4905-93e4-8f0ecfb66d64"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:48.175274 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.175062 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-config-volume" (OuterVolumeSpecName: "config-volume") pod "74863722-c1da-4905-93e4-8f0ecfb66d64" (UID: "74863722-c1da-4905-93e4-8f0ecfb66d64"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:48.175274 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.175088 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74863722-c1da-4905-93e4-8f0ecfb66d64-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "74863722-c1da-4905-93e4-8f0ecfb66d64" (UID: "74863722-c1da-4905-93e4-8f0ecfb66d64"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:59:48.175274 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.175203 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74863722-c1da-4905-93e4-8f0ecfb66d64-config-out" (OuterVolumeSpecName: "config-out") pod "74863722-c1da-4905-93e4-8f0ecfb66d64" (UID: "74863722-c1da-4905-93e4-8f0ecfb66d64"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:59:48.175625 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.175595 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "74863722-c1da-4905-93e4-8f0ecfb66d64" (UID: "74863722-c1da-4905-93e4-8f0ecfb66d64"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:48.175762 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.175734 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "74863722-c1da-4905-93e4-8f0ecfb66d64" (UID: "74863722-c1da-4905-93e4-8f0ecfb66d64"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:48.176972 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.176937 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74863722-c1da-4905-93e4-8f0ecfb66d64-kube-api-access-w9pzp" (OuterVolumeSpecName: "kube-api-access-w9pzp") pod "74863722-c1da-4905-93e4-8f0ecfb66d64" (UID: "74863722-c1da-4905-93e4-8f0ecfb66d64"). InnerVolumeSpecName "kube-api-access-w9pzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:59:48.178262 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.178235 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "74863722-c1da-4905-93e4-8f0ecfb66d64" (UID: "74863722-c1da-4905-93e4-8f0ecfb66d64"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:48.186840 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.186808 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-web-config" (OuterVolumeSpecName: "web-config") pod "74863722-c1da-4905-93e4-8f0ecfb66d64" (UID: "74863722-c1da-4905-93e4-8f0ecfb66d64"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:48.265395 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.265355 2561 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-cluster-tls-config\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:48.265395 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.265388 2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-web-config\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:48.265395 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.265397 2561 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-config-volume\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:48.265633 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.265407 2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74863722-c1da-4905-93e4-8f0ecfb66d64-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:48.265633 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.265417 2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/74863722-c1da-4905-93e4-8f0ecfb66d64-alertmanager-main-db\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:48.265633 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.265425 2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74863722-c1da-4905-93e4-8f0ecfb66d64-config-out\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:48.265633 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.265433 2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74863722-c1da-4905-93e4-8f0ecfb66d64-tls-assets\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:48.265633 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.265440 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9pzp\" (UniqueName: \"kubernetes.io/projected/74863722-c1da-4905-93e4-8f0ecfb66d64-kube-api-access-w9pzp\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:48.265633 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.265449 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:48.265633 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.265457 2561 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74863722-c1da-4905-93e4-8f0ecfb66d64-metrics-client-ca\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:48.265633 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.265480 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-main-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:48.265633 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.265489 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:48.265633 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.265524 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/74863722-c1da-4905-93e4-8f0ecfb66d64-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:48.925609 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.925572 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74863722-c1da-4905-93e4-8f0ecfb66d64","Type":"ContainerDied","Data":"7f3c8618c6dcee7e6fa0218f741871782578599e44c18cf91fac237af6e8d120"} Jul 31 22:59:48.926077 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.925633 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:48.926077 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.925648 2561 scope.go:117] "RemoveContainer" containerID="0d6db748d198ffe2c3ce5c46bc95a3442e3ba02203d1e5babe4099f8570fd534" Jul 31 22:59:48.936676 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.934867 2561 scope.go:117] "RemoveContainer" containerID="e911f5437ee76a1002e5f4479e62611fa77c914635f2d86bebf629ccee8cf02f" Jul 31 22:59:48.945715 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.945691 2561 scope.go:117] "RemoveContainer" containerID="174cc57f7c58e589593d0d605b122247c05992146fc56044db5f4f891d085eec" Jul 31 22:59:48.952444 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.952425 2561 scope.go:117] "RemoveContainer" containerID="eed2a6a5ba9f219ee41455fdc478c36060b6b6cb011cffebdf84a1e2ed547c41" Jul 31 22:59:48.957676 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.957648 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 22:59:48.960705 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.960669 2561 scope.go:117] "RemoveContainer" containerID="2f7e592b0f0a89b88af44775b4b59656852deb90300e1dd7e6727ca2f0647271" Jul 31 22:59:48.962376 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.962358 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 22:59:48.967974 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.967955 2561 scope.go:117] "RemoveContainer" containerID="7d4c72bc6568ac8c17956d28939d96ea45c6c7e0e097171c703770cad3d72e0c" Jul 31 22:59:48.975096 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.975080 2561 scope.go:117] "RemoveContainer" containerID="e7a3d60f189c5d304c5a51123b7535414b71b0cbc97b07a2b05060ffa9785c4b" Jul 31 22:59:48.990250 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990228 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 22:59:48.990571 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990558 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="kube-rbac-proxy" Jul 31 22:59:48.990620 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990572 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="kube-rbac-proxy" Jul 31 22:59:48.990620 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990584 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="alertmanager" Jul 31 22:59:48.990620 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990590 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="alertmanager" Jul 31 22:59:48.990620 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990597 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="init-config-reloader" Jul 31 22:59:48.990620 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990603 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="init-config-reloader" Jul 31 22:59:48.990620 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990617 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="kube-rbac-proxy-metric" Jul 31 22:59:48.990620 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990623 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="kube-rbac-proxy-metric" Jul 31 22:59:48.990818 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990629 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="prom-label-proxy" Jul 31 22:59:48.990818 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990634 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="prom-label-proxy" Jul 31 22:59:48.990818 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990643 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="kube-rbac-proxy-web" Jul 31 22:59:48.990818 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990648 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="kube-rbac-proxy-web" Jul 31 22:59:48.990818 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990656 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="config-reloader" Jul 31 22:59:48.990818 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990661 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="config-reloader" Jul 31 22:59:48.990818 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990715 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="config-reloader" Jul 31 22:59:48.990818 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990722 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="prom-label-proxy" Jul 31 22:59:48.990818 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990729 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="kube-rbac-proxy-web" Jul 31 22:59:48.990818 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990735 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="alertmanager" Jul 31 22:59:48.990818 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990741 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="kube-rbac-proxy-metric" Jul 31 22:59:48.990818 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.990748 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" containerName="kube-rbac-proxy" Jul 31 22:59:48.994155 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.994137 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:48.996977 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.996958 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 31 22:59:48.996977 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.996972 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 31 22:59:48.997135 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.997081 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 31 22:59:48.997135 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.997099 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 31 22:59:48.997240 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.997230 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 31 22:59:48.997290 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.997271 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 31 22:59:48.997345 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.997337 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rdj2n\"" Jul 31 22:59:48.997666 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.997649 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 31 22:59:48.997750 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:48.997719 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 31 22:59:49.004044 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.004024 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 31 22:59:49.011436 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.011409 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 22:59:49.173268 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.173157 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a693b69-6296-4761-a114-7b85f033b766-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.173268 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.173219 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.173268 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.173262 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a693b69-6296-4761-a114-7b85f033b766-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.173613 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.173303 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.173613 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.173340 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-config-volume\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.173613 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.173357 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a693b69-6296-4761-a114-7b85f033b766-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.173613 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.173386 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a693b69-6296-4761-a114-7b85f033b766-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.173613 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.173450 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.173613 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.173531 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.173613 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.173564 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qnxk\" (UniqueName: \"kubernetes.io/projected/0a693b69-6296-4761-a114-7b85f033b766-kube-api-access-5qnxk\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.173613 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.173591 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.173613 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.173616 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-web-config\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.173912 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.173639 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a693b69-6296-4761-a114-7b85f033b766-config-out\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.274322 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.274268 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.274563 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.274341 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-config-volume\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.274563 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.274366 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a693b69-6296-4761-a114-7b85f033b766-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.274563 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.274387 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a693b69-6296-4761-a114-7b85f033b766-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.274563 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.274411 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.274563 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.274439 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.274563 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.274467 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qnxk\" (UniqueName: \"kubernetes.io/projected/0a693b69-6296-4761-a114-7b85f033b766-kube-api-access-5qnxk\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.274563 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.274496 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.274563 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.274547 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-web-config\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.274968 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.274656 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a693b69-6296-4761-a114-7b85f033b766-config-out\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.274968 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.274699 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a693b69-6296-4761-a114-7b85f033b766-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.274968 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.274749 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.274968 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.274782 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a693b69-6296-4761-a114-7b85f033b766-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.275167 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.275138 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a693b69-6296-4761-a114-7b85f033b766-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.275307 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.275282 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a693b69-6296-4761-a114-7b85f033b766-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.277845 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.277566 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.277845 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.277566 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-config-volume\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.277845 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.277762 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a693b69-6296-4761-a114-7b85f033b766-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.277845 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.277783 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.277845 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.277799 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a693b69-6296-4761-a114-7b85f033b766-config-out\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.277845 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.277820 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.278219 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.277971 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.278219 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.278198 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a693b69-6296-4761-a114-7b85f033b766-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.278219 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.278202 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-web-config\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.279885 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.279865 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a693b69-6296-4761-a114-7b85f033b766-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.282889 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.282865 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qnxk\" (UniqueName: \"kubernetes.io/projected/0a693b69-6296-4761-a114-7b85f033b766-kube-api-access-5qnxk\") pod \"alertmanager-main-0\" (UID: \"0a693b69-6296-4761-a114-7b85f033b766\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.305812 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.305767 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 31 22:59:49.447882 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.447859 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 22:59:49.450192 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:59:49.450165 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a693b69_6296_4761_a114_7b85f033b766.slice/crio-e0de271f2bda682c008eebcd9cb5aee17061ecdbc4d65352b86768e12bb94321 WatchSource:0}: Error finding container e0de271f2bda682c008eebcd9cb5aee17061ecdbc4d65352b86768e12bb94321: Status 404 returned error can't find the container with id e0de271f2bda682c008eebcd9cb5aee17061ecdbc4d65352b86768e12bb94321 Jul 31 22:59:49.930954 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.930917 2561 generic.go:356] "Generic (PLEG): container finished" podID="0a693b69-6296-4761-a114-7b85f033b766" containerID="ca5f39b5a096e6bcff04f0fe6d680bd7aee173ed98a3af3d8cf5b40185569d49" exitCode=0 Jul 31 22:59:49.931375 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.931001 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a693b69-6296-4761-a114-7b85f033b766","Type":"ContainerDied","Data":"ca5f39b5a096e6bcff04f0fe6d680bd7aee173ed98a3af3d8cf5b40185569d49"} Jul 31 22:59:49.931375 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:49.931075 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a693b69-6296-4761-a114-7b85f033b766","Type":"ContainerStarted","Data":"e0de271f2bda682c008eebcd9cb5aee17061ecdbc4d65352b86768e12bb94321"} Jul 31 22:59:50.133026 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.132992 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74863722-c1da-4905-93e4-8f0ecfb66d64" path="/var/lib/kubelet/pods/74863722-c1da-4905-93e4-8f0ecfb66d64/volumes" Jul 31 22:59:50.746930 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.746896 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9"] Jul 31 22:59:50.755577 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.755089 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.758438 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.758407 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 31 22:59:50.758617 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.758442 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 31 22:59:50.758617 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.758442 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 31 22:59:50.758805 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.758785 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 31 22:59:50.758805 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.758792 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 31 22:59:50.758975 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.758797 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-ps7k5\"" Jul 31 22:59:50.764613 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.764587 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 31 22:59:50.770836 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.770807 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9"] Jul 31 22:59:50.892068 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.892000 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.892248 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.892117 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-metrics-client-ca\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.892248 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.892168 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-serving-certs-ca-bundle\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.892248 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.892195 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-secret-telemeter-client\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.892248 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.892223 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-telemeter-client-tls\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.892490 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.892286 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvhkj\" (UniqueName: \"kubernetes.io/projected/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-kube-api-access-vvhkj\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.892490 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.892331 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-federate-client-tls\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.892490 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.892353 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.941165 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.940408 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a693b69-6296-4761-a114-7b85f033b766","Type":"ContainerStarted","Data":"147e592968961d2383f771be19becc71608561a8fcc20c60c34524ce53de4c51"} Jul 31 22:59:50.941165 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.940496 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a693b69-6296-4761-a114-7b85f033b766","Type":"ContainerStarted","Data":"6794ec2e5e561525260c87c51721b5cdf5b736365b7837a937affb4c6aa8d81f"} Jul 31 22:59:50.941165 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.940561 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a693b69-6296-4761-a114-7b85f033b766","Type":"ContainerStarted","Data":"cce39cc3c4666fb1b4584a15fdc2bf7c9d799ac4a77a41ed02eeb486fe6f4a01"} Jul 31 22:59:50.941165 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.940597 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a693b69-6296-4761-a114-7b85f033b766","Type":"ContainerStarted","Data":"ac51ae4134ca4d90defa09bdb9e843a1fee5e10165f170a8264f962c07133193"} Jul 31 22:59:50.941165 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.940636 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a693b69-6296-4761-a114-7b85f033b766","Type":"ContainerStarted","Data":"6b32f8975813445c4bacd4d4bd093952d9ae2111e5dda7fc8da393055cf2985b"} Jul 31 22:59:50.941165 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.940671 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a693b69-6296-4761-a114-7b85f033b766","Type":"ContainerStarted","Data":"68fe2183b4a7357feb12a0522cb32493b4380a2d41de4f5d981b1298d74f7224"} Jul 31 22:59:50.976216 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.975640 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.96016937 podStartE2EDuration="2.975617152s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.447782ms" isStatefulPod=false podCreationTimestamp="2026-07-31 22:59:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:59:50.973243362 +0000 UTC m=+147.445687569" watchObservedRunningTime="2026-07-31 22:59:50.975617152 +0000 UTC m=+147.448061337" Jul 31 22:59:50.993571 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.993533 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvhkj\" (UniqueName: \"kubernetes.io/projected/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-kube-api-access-vvhkj\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.993767 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.993624 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-federate-client-tls\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.993767 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.993652 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.993884 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.993792 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.993945 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.993894 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-metrics-client-ca\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.994001 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.993945 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-serving-certs-ca-bundle\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.994001 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.993970 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-secret-telemeter-client\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.994091 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.994014 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-telemeter-client-tls\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.995481 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.994712 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.995481 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.995177 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-metrics-client-ca\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.995841 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.995814 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-serving-certs-ca-bundle\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.997336 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.997257 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.997839 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.997819 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-federate-client-tls\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.997943 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.997869 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-secret-telemeter-client\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:50.998079 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:50.998042 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-telemeter-client-tls\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:51.003375 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.003344 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvhkj\" (UniqueName: \"kubernetes.io/projected/6c512b95-63f0-4dfb-9bd9-d0f23615e4d4-kube-api-access-vvhkj\") pod \"telemeter-client-6c8f4cb785-vhmr9\" (UID: \"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4\") " pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:51.067798 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.067761 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" Jul 31 22:59:51.091533 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.090775 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 31 22:59:51.091533 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.091172 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="prometheus" containerID="cri-o://fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04" gracePeriod=600 Jul 31 22:59:51.091765 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.091742 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="kube-rbac-proxy" containerID="cri-o://6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976" gracePeriod=600 Jul 31 22:59:51.092003 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.091852 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="thanos-sidecar" containerID="cri-o://ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32" gracePeriod=600 Jul 31 22:59:51.092003 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.091893 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="config-reloader" containerID="cri-o://70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852" gracePeriod=600 Jul 31 22:59:51.092003 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.091927 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1" gracePeriod=600 Jul 31 22:59:51.092179 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.092001 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="kube-rbac-proxy-web" containerID="cri-o://518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca" gracePeriod=600 Jul 31 22:59:51.246156 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.246123 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9"] Jul 31 22:59:51.250702 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:59:51.250667 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c512b95_63f0_4dfb_9bd9_d0f23615e4d4.slice/crio-112376a43f125edad8ab63125ae07c921120c7b2538fcf0efd4247261272aba6 WatchSource:0}: Error finding container 112376a43f125edad8ab63125ae07c921120c7b2538fcf0efd4247261272aba6: Status 404 returned error can't find the container with id 112376a43f125edad8ab63125ae07c921120c7b2538fcf0efd4247261272aba6 Jul 31 22:59:51.355332 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.355306 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:51.398135 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398104 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398135 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398141 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-kube-rbac-proxy\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398381 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398176 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-1\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398381 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398200 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-trusted-ca-bundle\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398381 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398226 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-0\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398381 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398257 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-tls\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398381 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398280 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-config\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398381 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398300 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-metrics-client-certs\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398381 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398335 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-tls-assets\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398381 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398367 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-config-out\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398780 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398393 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2mwm\" (UniqueName: \"kubernetes.io/projected/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-kube-api-access-c2mwm\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398780 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398423 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398780 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398456 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-thanos-prometheus-http-client-file\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398780 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398522 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-metrics-client-ca\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398780 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398551 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-serving-certs-ca-bundle\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398780 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398578 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-grpc-tls\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398780 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398607 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-db\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398780 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398648 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-2\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.398780 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.398688 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-kubelet-serving-ca-bundle\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.399453 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.399298 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:59:51.399796 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.399733 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:59:51.400311 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.400031 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:59:51.400396 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.400341 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:59:51.400750 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.400606 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:59:51.400842 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.400751 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:59:51.401622 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.401593 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:59:51.403746 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.403708 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:59:51.404253 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.404224 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:51.406205 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.405994 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:51.406205 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.406114 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-config" (OuterVolumeSpecName: "config") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:51.406205 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.406184 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-config-out" (OuterVolumeSpecName: "config-out") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:59:51.406513 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.406338 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:51.407359 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.407326 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-kube-api-access-c2mwm" (OuterVolumeSpecName: "kube-api-access-c2mwm") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "kube-api-access-c2mwm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:59:51.407448 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.407411 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:51.407778 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.407658 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:59:51.407778 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.407660 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:51.407778 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.407743 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:51.408902 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.408879 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:51.499801 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.499714 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-web-config\") pod \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\" (UID: \"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4\") " Jul 31 22:59:51.499998 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.499982 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-metrics-client-ca\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500035 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500006 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500035 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500024 2561 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-grpc-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500102 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500039 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-db\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500102 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500069 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500102 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500078 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500102 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500089 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500102 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500099 2561 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-kube-rbac-proxy\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500245 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500108 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500245 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500117 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500245 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500125 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500245 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500135 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500245 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500144 2561 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-config\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500245 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500152 2561 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-metrics-client-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500245 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500161 2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-tls-assets\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500245 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500170 2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-config-out\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500245 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500177 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2mwm\" (UniqueName: \"kubernetes.io/projected/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-kube-api-access-c2mwm\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500245 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500186 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.500245 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.500211 2561 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.511836 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.511808 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-web-config" (OuterVolumeSpecName: "web-config") pod "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" (UID: "ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:59:51.600607 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.600572 2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4-web-config\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 22:59:51.948800 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.948766 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" event={"ID":"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4","Type":"ContainerStarted","Data":"112376a43f125edad8ab63125ae07c921120c7b2538fcf0efd4247261272aba6"} Jul 31 22:59:51.951897 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.951872 2561 generic.go:356] "Generic (PLEG): container finished" podID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerID="3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1" exitCode=0 Jul 31 22:59:51.951897 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.951894 2561 generic.go:356] "Generic (PLEG): container finished" podID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerID="6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976" exitCode=0 Jul 31 22:59:51.951897 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.951902 2561 generic.go:356] "Generic (PLEG): container finished" podID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerID="518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca" exitCode=0 Jul 31 22:59:51.951897 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.951908 2561 generic.go:356] "Generic (PLEG): container finished" podID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerID="ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32" exitCode=0 Jul 31 22:59:51.952131 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.951913 2561 generic.go:356] "Generic (PLEG): container finished" podID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerID="70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852" exitCode=0 Jul 31 22:59:51.952131 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.951919 2561 generic.go:356] "Generic (PLEG): container finished" podID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerID="fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04" exitCode=0 Jul 31 22:59:51.952131 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.951951 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:51.952131 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.951957 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4","Type":"ContainerDied","Data":"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1"} Jul 31 22:59:51.952131 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.952015 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4","Type":"ContainerDied","Data":"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976"} Jul 31 22:59:51.952131 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.952043 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4","Type":"ContainerDied","Data":"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca"} Jul 31 22:59:51.952131 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.952069 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4","Type":"ContainerDied","Data":"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32"} Jul 31 22:59:51.952131 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.952081 2561 scope.go:117] "RemoveContainer" containerID="3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1" Jul 31 22:59:51.952131 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.952092 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4","Type":"ContainerDied","Data":"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852"} Jul 31 22:59:51.952131 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.952118 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4","Type":"ContainerDied","Data":"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04"} Jul 31 22:59:51.952529 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.952142 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4","Type":"ContainerDied","Data":"83ee144bd652e59f1ba6c855b1cfd2dec971ebaff46eb55d31062811a27b1977"} Jul 31 22:59:51.960275 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.960067 2561 scope.go:117] "RemoveContainer" containerID="6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976" Jul 31 22:59:51.967424 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.967397 2561 scope.go:117] "RemoveContainer" containerID="518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca" Jul 31 22:59:51.974454 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.974429 2561 scope.go:117] "RemoveContainer" containerID="ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32" Jul 31 22:59:51.976973 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.976952 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 31 22:59:51.984538 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.983977 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 31 22:59:51.988211 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.988188 2561 scope.go:117] "RemoveContainer" containerID="70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852" Jul 31 22:59:51.996967 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:51.996873 2561 scope.go:117] "RemoveContainer" containerID="fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04" Jul 31 22:59:52.006871 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.006851 2561 scope.go:117] "RemoveContainer" containerID="429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa" Jul 31 22:59:52.014813 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.014784 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 31 22:59:52.015292 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015271 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="kube-rbac-proxy" Jul 31 22:59:52.015292 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015293 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="kube-rbac-proxy" Jul 31 22:59:52.015538 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015314 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="init-config-reloader" Jul 31 22:59:52.015538 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015322 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="init-config-reloader" Jul 31 22:59:52.015538 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015344 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="thanos-sidecar" Jul 31 22:59:52.015538 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015353 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="thanos-sidecar" Jul 31 22:59:52.015538 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015366 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="config-reloader" Jul 31 22:59:52.015538 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015375 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="config-reloader" Jul 31 22:59:52.015538 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015388 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="kube-rbac-proxy-web" Jul 31 22:59:52.015538 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015397 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="kube-rbac-proxy-web" Jul 31 22:59:52.015538 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015418 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="kube-rbac-proxy-thanos" Jul 31 22:59:52.015538 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015428 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="kube-rbac-proxy-thanos" Jul 31 22:59:52.015538 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015439 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="prometheus" Jul 31 22:59:52.015538 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015447 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="prometheus" Jul 31 22:59:52.015538 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015539 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="kube-rbac-proxy" Jul 31 22:59:52.016189 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015553 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="kube-rbac-proxy-web" Jul 31 22:59:52.016189 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015560 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="config-reloader" Jul 31 22:59:52.016189 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015566 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="thanos-sidecar" Jul 31 22:59:52.016189 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015573 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="prometheus" Jul 31 22:59:52.016189 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015579 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" containerName="kube-rbac-proxy-thanos" Jul 31 22:59:52.016189 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.015901 2561 scope.go:117] "RemoveContainer" containerID="3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1" Jul 31 22:59:52.016968 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:59:52.016474 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1\": container with ID starting with 3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1 not found: ID does not exist" containerID="3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1" Jul 31 22:59:52.016968 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.016532 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1"} err="failed to get container status \"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1\": rpc error: code = NotFound desc = could not find container \"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1\": container with ID starting with 3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1 not found: ID does not exist" Jul 31 22:59:52.016968 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.016576 2561 scope.go:117] "RemoveContainer" containerID="6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976" Jul 31 22:59:52.017215 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:59:52.017189 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976\": container with ID starting with 6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976 not found: ID does not exist" containerID="6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976" Jul 31 22:59:52.017326 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.017221 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976"} err="failed to get container status \"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976\": rpc error: code = NotFound desc = could not find container \"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976\": container with ID starting with 6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976 not found: ID does not exist" Jul 31 22:59:52.017326 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.017243 2561 scope.go:117] "RemoveContainer" containerID="518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca" Jul 31 22:59:52.018797 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:59:52.018695 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca\": container with ID starting with 518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca not found: ID does not exist" containerID="518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca" Jul 31 22:59:52.018797 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.018730 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca"} err="failed to get container status \"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca\": rpc error: code = NotFound desc = could not find container \"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca\": container with ID starting with 518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca not found: ID does not exist" Jul 31 22:59:52.018797 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.018753 2561 scope.go:117] "RemoveContainer" containerID="ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32" Jul 31 22:59:52.019230 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:59:52.019204 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32\": container with ID starting with ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32 not found: ID does not exist" containerID="ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32" Jul 31 22:59:52.019321 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.019236 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32"} err="failed to get container status \"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32\": rpc error: code = NotFound desc = could not find container \"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32\": container with ID starting with ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32 not found: ID does not exist" Jul 31 22:59:52.019321 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.019257 2561 scope.go:117] "RemoveContainer" containerID="70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852" Jul 31 22:59:52.019786 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:59:52.019745 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852\": container with ID starting with 70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852 not found: ID does not exist" containerID="70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852" Jul 31 22:59:52.019786 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.019777 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852"} err="failed to get container status \"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852\": rpc error: code = NotFound desc = could not find container \"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852\": container with ID starting with 70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852 not found: ID does not exist" Jul 31 22:59:52.019913 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.019798 2561 scope.go:117] "RemoveContainer" containerID="fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04" Jul 31 22:59:52.020134 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:59:52.020109 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04\": container with ID starting with fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04 not found: ID does not exist" containerID="fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04" Jul 31 22:59:52.020282 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.020142 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04"} err="failed to get container status \"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04\": rpc error: code = NotFound desc = could not find container \"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04\": container with ID starting with fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04 not found: ID does not exist" Jul 31 22:59:52.020282 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.020165 2561 scope.go:117] "RemoveContainer" containerID="429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa" Jul 31 22:59:52.020483 ip-10-0-131-148 kubenswrapper[2561]: E0731 22:59:52.020458 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa\": container with ID starting with 429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa not found: ID does not exist" containerID="429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa" Jul 31 22:59:52.020640 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.020487 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa"} err="failed to get container status \"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa\": rpc error: code = NotFound desc = could not find container \"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa\": container with ID starting with 429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa not found: ID does not exist" Jul 31 22:59:52.020700 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.020644 2561 scope.go:117] "RemoveContainer" containerID="3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1" Jul 31 22:59:52.020700 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.020656 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.021706 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.021673 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1"} err="failed to get container status \"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1\": rpc error: code = NotFound desc = could not find container \"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1\": container with ID starting with 3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1 not found: ID does not exist" Jul 31 22:59:52.021785 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.021708 2561 scope.go:117] "RemoveContainer" containerID="6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976" Jul 31 22:59:52.021991 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.021961 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976"} err="failed to get container status \"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976\": rpc error: code = NotFound desc = could not find container \"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976\": container with ID starting with 6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976 not found: ID does not exist" Jul 31 22:59:52.021991 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.021988 2561 scope.go:117] "RemoveContainer" containerID="518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca" Jul 31 22:59:52.022299 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.022272 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca"} err="failed to get container status \"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca\": rpc error: code = NotFound desc = could not find container \"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca\": container with ID starting with 518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca not found: ID does not exist" Jul 31 22:59:52.022299 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.022298 2561 scope.go:117] "RemoveContainer" containerID="ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32" Jul 31 22:59:52.022572 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.022544 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32"} err="failed to get container status \"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32\": rpc error: code = NotFound desc = could not find container \"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32\": container with ID starting with ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32 not found: ID does not exist" Jul 31 22:59:52.022572 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.022573 2561 scope.go:117] "RemoveContainer" containerID="70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852" Jul 31 22:59:52.022802 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.022781 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852"} err="failed to get container status \"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852\": rpc error: code = NotFound desc = could not find container \"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852\": container with ID starting with 70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852 not found: ID does not exist" Jul 31 22:59:52.022858 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.022802 2561 scope.go:117] "RemoveContainer" containerID="fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04" Jul 31 22:59:52.023063 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.023042 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04"} err="failed to get container status \"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04\": rpc error: code = NotFound desc = could not find container \"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04\": container with ID starting with fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04 not found: ID does not exist" Jul 31 22:59:52.023149 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.023063 2561 scope.go:117] "RemoveContainer" containerID="429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa" Jul 31 22:59:52.023316 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.023297 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa"} err="failed to get container status \"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa\": rpc error: code = NotFound desc = could not find container \"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa\": container with ID starting with 429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa not found: ID does not exist" Jul 31 22:59:52.023387 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.023317 2561 scope.go:117] "RemoveContainer" containerID="3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1" Jul 31 22:59:52.023628 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.023603 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1"} err="failed to get container status \"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1\": rpc error: code = NotFound desc = could not find container \"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1\": container with ID starting with 3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1 not found: ID does not exist" Jul 31 22:59:52.023686 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.023631 2561 scope.go:117] "RemoveContainer" containerID="6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976" Jul 31 22:59:52.023894 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.023873 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976"} err="failed to get container status \"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976\": rpc error: code = NotFound desc = could not find container \"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976\": container with ID starting with 6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976 not found: ID does not exist" Jul 31 22:59:52.023966 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.023895 2561 scope.go:117] "RemoveContainer" containerID="518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca" Jul 31 22:59:52.024144 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.024126 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca"} err="failed to get container status \"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca\": rpc error: code = NotFound desc = could not find container \"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca\": container with ID starting with 518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca not found: ID does not exist" Jul 31 22:59:52.024215 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.024147 2561 scope.go:117] "RemoveContainer" containerID="ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32" Jul 31 22:59:52.024469 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.024444 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32"} err="failed to get container status \"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32\": rpc error: code = NotFound desc = could not find container \"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32\": container with ID starting with ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32 not found: ID does not exist" Jul 31 22:59:52.024548 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.024471 2561 scope.go:117] "RemoveContainer" containerID="70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852" Jul 31 22:59:52.024842 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.024817 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852"} err="failed to get container status \"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852\": rpc error: code = NotFound desc = could not find container \"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852\": container with ID starting with 70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852 not found: ID does not exist" Jul 31 22:59:52.024842 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.024843 2561 scope.go:117] "RemoveContainer" containerID="fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04" Jul 31 22:59:52.025109 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.025074 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04"} err="failed to get container status \"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04\": rpc error: code = NotFound desc = could not find container \"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04\": container with ID starting with fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04 not found: ID does not exist" Jul 31 22:59:52.025109 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.025094 2561 scope.go:117] "RemoveContainer" containerID="429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa" Jul 31 22:59:52.025754 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.025292 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa"} err="failed to get container status \"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa\": rpc error: code = NotFound desc = could not find container \"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa\": container with ID starting with 429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa not found: ID does not exist" Jul 31 22:59:52.025754 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.025317 2561 scope.go:117] "RemoveContainer" containerID="3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1" Jul 31 22:59:52.025754 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.025396 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 31 22:59:52.025754 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.025467 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 31 22:59:52.025754 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.025486 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 31 22:59:52.025754 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.025563 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 31 22:59:52.025754 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.025603 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1"} err="failed to get container status \"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1\": rpc error: code = NotFound desc = could not find container \"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1\": container with ID starting with 3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1 not found: ID does not exist" Jul 31 22:59:52.025754 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.025626 2561 scope.go:117] "RemoveContainer" containerID="6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976" Jul 31 22:59:52.025754 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.025640 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 31 22:59:52.025754 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.025740 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 31 22:59:52.026229 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.025894 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976"} err="failed to get container status \"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976\": rpc error: code = NotFound desc = could not find container \"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976\": container with ID starting with 6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976 not found: ID does not exist" Jul 31 22:59:52.026229 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.025918 2561 scope.go:117] "RemoveContainer" containerID="518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca" Jul 31 22:59:52.026229 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.026099 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 31 22:59:52.026229 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.026169 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca"} err="failed to get container status \"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca\": rpc error: code = NotFound desc = could not find container \"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca\": container with ID starting with 518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca not found: ID does not exist" Jul 31 22:59:52.026229 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.026189 2561 scope.go:117] "RemoveContainer" containerID="ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32" Jul 31 22:59:52.026465 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.026104 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 31 22:59:52.026465 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.026353 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 31 22:59:52.026465 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.026428 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-78r9ui4g8hkem\"" Jul 31 22:59:52.026656 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.026557 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32"} err="failed to get container status \"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32\": rpc error: code = NotFound desc = could not find container \"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32\": container with ID starting with ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32 not found: ID does not exist" Jul 31 22:59:52.026656 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.026572 2561 scope.go:117] "RemoveContainer" containerID="70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852" Jul 31 22:59:52.027102 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.027083 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 31 22:59:52.027102 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.027089 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852"} err="failed to get container status \"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852\": rpc error: code = NotFound desc = could not find container \"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852\": container with ID starting with 70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852 not found: ID does not exist" Jul 31 22:59:52.027243 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.027112 2561 scope.go:117] "RemoveContainer" containerID="fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04" Jul 31 22:59:52.027396 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.027370 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04"} err="failed to get container status \"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04\": rpc error: code = NotFound desc = could not find container \"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04\": container with ID starting with fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04 not found: ID does not exist" Jul 31 22:59:52.028265 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.027430 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nzcdr\"" Jul 31 22:59:52.028265 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.028152 2561 scope.go:117] "RemoveContainer" containerID="429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa" Jul 31 22:59:52.028265 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.027456 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 31 22:59:52.028477 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.027463 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 31 22:59:52.028894 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.028795 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa"} err="failed to get container status \"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa\": rpc error: code = NotFound desc = could not find container \"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa\": container with ID starting with 429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa not found: ID does not exist" Jul 31 22:59:52.028894 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.028821 2561 scope.go:117] "RemoveContainer" containerID="3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1" Jul 31 22:59:52.029265 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.029143 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1"} err="failed to get container status \"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1\": rpc error: code = NotFound desc = could not find container \"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1\": container with ID starting with 3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1 not found: ID does not exist" Jul 31 22:59:52.029265 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.029195 2561 scope.go:117] "RemoveContainer" containerID="6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976" Jul 31 22:59:52.029537 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.029395 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 31 22:59:52.029714 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.029686 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976"} err="failed to get container status \"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976\": rpc error: code = NotFound desc = could not find container \"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976\": container with ID starting with 6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976 not found: ID does not exist" Jul 31 22:59:52.029791 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.029717 2561 scope.go:117] "RemoveContainer" containerID="518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca" Jul 31 22:59:52.030132 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.030104 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca"} err="failed to get container status \"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca\": rpc error: code = NotFound desc = could not find container \"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca\": container with ID starting with 518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca not found: ID does not exist" Jul 31 22:59:52.030132 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.030131 2561 scope.go:117] "RemoveContainer" containerID="ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32" Jul 31 22:59:52.030419 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.030391 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32"} err="failed to get container status \"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32\": rpc error: code = NotFound desc = could not find container \"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32\": container with ID starting with ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32 not found: ID does not exist" Jul 31 22:59:52.030419 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.030417 2561 scope.go:117] "RemoveContainer" containerID="70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852" Jul 31 22:59:52.032043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.031987 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852"} err="failed to get container status \"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852\": rpc error: code = NotFound desc = could not find container \"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852\": container with ID starting with 70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852 not found: ID does not exist" Jul 31 22:59:52.032043 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.032015 2561 scope.go:117] "RemoveContainer" containerID="fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04" Jul 31 22:59:52.032382 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.032290 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04"} err="failed to get container status \"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04\": rpc error: code = NotFound desc = could not find container \"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04\": container with ID starting with fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04 not found: ID does not exist" Jul 31 22:59:52.032382 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.032317 2561 scope.go:117] "RemoveContainer" containerID="429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa" Jul 31 22:59:52.032596 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.032573 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa"} err="failed to get container status \"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa\": rpc error: code = NotFound desc = could not find container \"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa\": container with ID starting with 429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa not found: ID does not exist" Jul 31 22:59:52.032661 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.032602 2561 scope.go:117] "RemoveContainer" containerID="3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1" Jul 31 22:59:52.032891 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.032861 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1"} err="failed to get container status \"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1\": rpc error: code = NotFound desc = could not find container \"3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1\": container with ID starting with 3e81e0ba9dc8a4d3c74abacde5907ced24507ef7782aa589b873b519772081f1 not found: ID does not exist" Jul 31 22:59:52.032960 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.032893 2561 scope.go:117] "RemoveContainer" containerID="6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976" Jul 31 22:59:52.032960 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.032949 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 31 22:59:52.033168 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.033141 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976"} err="failed to get container status \"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976\": rpc error: code = NotFound desc = could not find container \"6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976\": container with ID starting with 6a3b084a14b08b39de1449886a5765f3dc84bf05d7883e7e9748a43bc916c976 not found: ID does not exist" Jul 31 22:59:52.033242 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.033166 2561 scope.go:117] "RemoveContainer" containerID="518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca" Jul 31 22:59:52.033424 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.033405 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca"} err="failed to get container status \"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca\": rpc error: code = NotFound desc = could not find container \"518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca\": container with ID starting with 518c90aab790bd2d50b5545f928b1f53e2ac6483145c9796369d03477dfb47ca not found: ID does not exist" Jul 31 22:59:52.033487 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.033423 2561 scope.go:117] "RemoveContainer" containerID="ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32" Jul 31 22:59:52.033743 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.033710 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32"} err="failed to get container status \"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32\": rpc error: code = NotFound desc = could not find container \"ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32\": container with ID starting with ff8391163c71530d33068f5cf28aec67473141113b73ecf34c9a3f7930227b32 not found: ID does not exist" Jul 31 22:59:52.033743 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.033738 2561 scope.go:117] "RemoveContainer" containerID="70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852" Jul 31 22:59:52.034030 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.034002 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852"} err="failed to get container status \"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852\": rpc error: code = NotFound desc = could not find container \"70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852\": container with ID starting with 70abe05649fec4d792787978094d59f8a6b2fc409b1b575a631db5c331036852 not found: ID does not exist" Jul 31 22:59:52.034030 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.034029 2561 scope.go:117] "RemoveContainer" containerID="fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04" Jul 31 22:59:52.034335 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.034315 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04"} err="failed to get container status \"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04\": rpc error: code = NotFound desc = could not find container \"fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04\": container with ID starting with fb8d0b284fa800f04b7078a1df09cfbcf772e743c25018c61594bac330d84e04 not found: ID does not exist" Jul 31 22:59:52.034335 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.034334 2561 scope.go:117] "RemoveContainer" containerID="429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa" Jul 31 22:59:52.034583 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.034561 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa"} err="failed to get container status \"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa\": rpc error: code = NotFound desc = could not find container \"429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa\": container with ID starting with 429413053a21629a09a260797ffa78ab84dfa457d08c5aabc52e309037dc76aa not found: ID does not exist" Jul 31 22:59:52.040685 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.040613 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 31 22:59:52.104126 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104038 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104126 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104077 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b212a90f-da06-473c-b8e0-68249246af87-config-out\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104126 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104096 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104400 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104146 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-web-config\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104400 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104183 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b212a90f-da06-473c-b8e0-68249246af87-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104400 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104210 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104400 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104236 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b212a90f-da06-473c-b8e0-68249246af87-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104400 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104266 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-config\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104400 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104318 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104400 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104349 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104400 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104373 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104400 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104398 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104822 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104439 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104822 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104471 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9gvv\" (UniqueName: \"kubernetes.io/projected/b212a90f-da06-473c-b8e0-68249246af87-kube-api-access-v9gvv\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104822 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104537 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104822 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104556 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104822 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104573 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104822 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104593 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104822 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104616 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.104822 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.104698 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.134333 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.134299 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4" path="/var/lib/kubelet/pods/ea6c3200-1f3d-437b-9d52-26c4c3b5a4a4/volumes" Jul 31 22:59:52.205150 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.205108 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-config\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.205332 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.205195 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.205332 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.205233 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.205332 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.205261 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.205550 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.205476 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.205609 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.205558 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.205747 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9gvv\" (UniqueName: \"kubernetes.io/projected/b212a90f-da06-473c-b8e0-68249246af87-kube-api-access-v9gvv\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.205795 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.205823 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.205850 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.205880 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.205915 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.205977 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.206013 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.206065 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b212a90f-da06-473c-b8e0-68249246af87-config-out\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.206092 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.206120 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-web-config\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.206136 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.206143 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b212a90f-da06-473c-b8e0-68249246af87-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.206206 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.206209 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.206461 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.206242 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b212a90f-da06-473c-b8e0-68249246af87-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.207405 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.206818 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.207405 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.206919 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.208789 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.207742 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.208789 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.208383 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-config\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.208964 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.208935 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b212a90f-da06-473c-b8e0-68249246af87-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.209202 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.209176 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.209820 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.209794 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.210003 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.209980 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b212a90f-da06-473c-b8e0-68249246af87-config-out\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.210787 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.210744 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.211849 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.211825 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b212a90f-da06-473c-b8e0-68249246af87-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.212532 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.212457 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.212532 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.212459 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.212651 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.212613 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.212703 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.212650 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-web-config\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.212703 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.212670 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.213295 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.213252 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b212a90f-da06-473c-b8e0-68249246af87-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.214684 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.214656 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b212a90f-da06-473c-b8e0-68249246af87-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.219263 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.219240 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9gvv\" (UniqueName: \"kubernetes.io/projected/b212a90f-da06-473c-b8e0-68249246af87-kube-api-access-v9gvv\") pod \"prometheus-k8s-0\" (UID: \"b212a90f-da06-473c-b8e0-68249246af87\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.333494 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.333456 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 22:59:52.477283 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.477216 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 31 22:59:52.480070 ip-10-0-131-148 kubenswrapper[2561]: W0731 22:59:52.480033 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb212a90f_da06_473c_b8e0_68249246af87.slice/crio-3faa3a30c57bbfbc4afe07fe45013c56d969982e86ba8d3752debf2a8c9e6039 WatchSource:0}: Error finding container 3faa3a30c57bbfbc4afe07fe45013c56d969982e86ba8d3752debf2a8c9e6039: Status 404 returned error can't find the container with id 3faa3a30c57bbfbc4afe07fe45013c56d969982e86ba8d3752debf2a8c9e6039 Jul 31 22:59:52.959593 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.959554 2561 generic.go:356] "Generic (PLEG): container finished" podID="b212a90f-da06-473c-b8e0-68249246af87" containerID="635498a2fb755c3d0a3ac8b77fc33febcf760a07580c9c21987d2b0ee8222bd2" exitCode=0 Jul 31 22:59:52.960032 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.959617 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b212a90f-da06-473c-b8e0-68249246af87","Type":"ContainerDied","Data":"635498a2fb755c3d0a3ac8b77fc33febcf760a07580c9c21987d2b0ee8222bd2"} Jul 31 22:59:52.960032 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:52.959708 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b212a90f-da06-473c-b8e0-68249246af87","Type":"ContainerStarted","Data":"3faa3a30c57bbfbc4afe07fe45013c56d969982e86ba8d3752debf2a8c9e6039"} Jul 31 22:59:53.967388 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:53.967344 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b212a90f-da06-473c-b8e0-68249246af87","Type":"ContainerStarted","Data":"d7b1ee57633bf0b113195a927eef736fbc6776f75be6379b48d73dd6e28a45f6"} Jul 31 22:59:53.967854 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:53.967438 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b212a90f-da06-473c-b8e0-68249246af87","Type":"ContainerStarted","Data":"ddd518928de6298ff4ef9a74c9e6e66e7d57584173a88eea584fde4a5ca95fa1"} Jul 31 22:59:53.967854 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:53.967476 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b212a90f-da06-473c-b8e0-68249246af87","Type":"ContainerStarted","Data":"9575983fa4d326f155498b1795278a5f3a491d5baafdf344ee7d83e577841953"} Jul 31 22:59:53.967854 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:53.967527 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b212a90f-da06-473c-b8e0-68249246af87","Type":"ContainerStarted","Data":"cdc68f81aa46c66eb6eb3cb16d092fa388dc2872de6acbb8aef8c754eac0b633"} Jul 31 22:59:53.967854 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:53.967567 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b212a90f-da06-473c-b8e0-68249246af87","Type":"ContainerStarted","Data":"6b47bfe7f81384c82552b697f7bb6c3558a8eafa4dafe364ded350d6f94fe13e"} Jul 31 22:59:53.967854 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:53.967602 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b212a90f-da06-473c-b8e0-68249246af87","Type":"ContainerStarted","Data":"b2f1e11c2d1d49d5d84360917ad0ae4407e2121195253c0dbc9a071c11cbb2ca"} Jul 31 22:59:53.969518 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:53.969482 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" event={"ID":"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4","Type":"ContainerStarted","Data":"74e78e37fa537c61cc091f194575d7003dbee19314ad951f187138b76b63417e"} Jul 31 22:59:53.969625 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:53.969539 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" event={"ID":"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4","Type":"ContainerStarted","Data":"2939852a263e15dc63f4fc191cfbe38ad00870ff9ef0cfd7fa15bb299fbacaf2"} Jul 31 22:59:53.969625 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:53.969558 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" event={"ID":"6c512b95-63f0-4dfb-9bd9-d0f23615e4d4","Type":"ContainerStarted","Data":"6aefbac7143467c294fa8df8a1435bd6fd8ae59e715daae44cad30acaa88533d"} Jul 31 22:59:54.000610 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:54.000539 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.978042236 podStartE2EDuration="3.000515244s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.473008ms" isStatefulPod=false podCreationTimestamp="2026-07-31 22:59:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:59:53.998324304 +0000 UTC m=+150.470768488" watchObservedRunningTime="2026-07-31 22:59:54.000515244 +0000 UTC m=+150.472959419" Jul 31 22:59:54.022328 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:54.022273 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6c8f4cb785-vhmr9" podStartSLOduration=2.109037812 podStartE2EDuration="4.022254157s" totalImagesPullingTime="1.913216345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:59:54.020612573 +0000 UTC m=+150.493056781" watchObservedRunningTime="2026-07-31 22:59:54.022254157 +0000 UTC m=+150.494698340" Jul 31 22:59:54.189268 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:54.189228 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:54.260204 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:54.260128 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 22:59:57.334258 ip-10-0-131-148 kubenswrapper[2561]: I0731 22:59:57.334228 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:00:04.701985 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:00:04.701949 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:00:04.763012 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:00:04.762975 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:00:20.112779 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:00:20.112694 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:00:52.334393 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:00:52.334333 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:00:52.350265 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:00:52.350237 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:00:53.222137 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:00:53.222109 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:01:00.931811 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:01:00.931770 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:01:00.995640 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:01:00.995610 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:01:12.181132 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:01:12.181094 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:01:12.249607 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:01:12.249570 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:01:12.978751 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:01:12.978711 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:01:13.041745 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:01:13.041716 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:01:16.041895 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:01:16.041856 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:01:16.142541 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:01:16.142512 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:01:22.887607 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:01:22.887570 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:01:22.988370 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:01:22.988331 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:01:33.664653 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:01:33.664614 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:00.529307 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:00.529274 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:00.626924 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:00.626889 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:22.808777 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:22.808741 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:23.960903 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:23.960870 2561 kubelet.go:1648] "Image garbage collection succeeded" Jul 31 23:02:24.271487 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:24.271310 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:26.745678 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:26.745638 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:26.845072 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:26.845017 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:28.297832 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:28.297797 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:28.419674 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:28.419639 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:28.532997 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:28.532952 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:30.464697 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:30.464665 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:31.109131 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.109101 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:31.456368 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.456289 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5rwf8"] Jul 31 23:02:31.459417 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.459396 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5rwf8" Jul 31 23:02:31.462069 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.462049 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-qm8hn\"" Jul 31 23:02:31.462179 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.462094 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 31 23:02:31.462179 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.462101 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 31 23:02:31.471163 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.471142 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5rwf8"] Jul 31 23:02:31.621756 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.621722 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48f171b4-a19f-4599-ab23-ce6dd9e968c0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5rwf8\" (UID: \"48f171b4-a19f-4599-ab23-ce6dd9e968c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5rwf8" Jul 31 23:02:31.621954 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.621789 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmglp\" (UniqueName: \"kubernetes.io/projected/48f171b4-a19f-4599-ab23-ce6dd9e968c0-kube-api-access-xmglp\") pod \"cert-manager-cainjector-559cdd8c87-5rwf8\" (UID: \"48f171b4-a19f-4599-ab23-ce6dd9e968c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5rwf8" Jul 31 23:02:31.723111 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.723014 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48f171b4-a19f-4599-ab23-ce6dd9e968c0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5rwf8\" (UID: \"48f171b4-a19f-4599-ab23-ce6dd9e968c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5rwf8" Jul 31 23:02:31.723111 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.723102 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmglp\" (UniqueName: \"kubernetes.io/projected/48f171b4-a19f-4599-ab23-ce6dd9e968c0-kube-api-access-xmglp\") pod \"cert-manager-cainjector-559cdd8c87-5rwf8\" (UID: \"48f171b4-a19f-4599-ab23-ce6dd9e968c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5rwf8" Jul 31 23:02:31.732344 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.732308 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48f171b4-a19f-4599-ab23-ce6dd9e968c0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5rwf8\" (UID: \"48f171b4-a19f-4599-ab23-ce6dd9e968c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5rwf8" Jul 31 23:02:31.732486 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.732421 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmglp\" (UniqueName: \"kubernetes.io/projected/48f171b4-a19f-4599-ab23-ce6dd9e968c0-kube-api-access-xmglp\") pod \"cert-manager-cainjector-559cdd8c87-5rwf8\" (UID: \"48f171b4-a19f-4599-ab23-ce6dd9e968c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5rwf8" Jul 31 23:02:31.780443 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.780409 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5rwf8" Jul 31 23:02:31.906179 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.906153 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5rwf8"] Jul 31 23:02:31.909000 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:02:31.908970 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48f171b4_a19f_4599_ab23_ce6dd9e968c0.slice/crio-8afdfa9ff4a8b9629dc779c98d60baaa462c7f5a9ddbb83357e592ae05ddc875 WatchSource:0}: Error finding container 8afdfa9ff4a8b9629dc779c98d60baaa462c7f5a9ddbb83357e592ae05ddc875: Status 404 returned error can't find the container with id 8afdfa9ff4a8b9629dc779c98d60baaa462c7f5a9ddbb83357e592ae05ddc875 Jul 31 23:02:31.910806 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:31.910789 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 31 23:02:32.605596 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:32.605563 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5rwf8" event={"ID":"48f171b4-a19f-4599-ab23-ce6dd9e968c0","Type":"ContainerStarted","Data":"8afdfa9ff4a8b9629dc779c98d60baaa462c7f5a9ddbb83357e592ae05ddc875"} Jul 31 23:02:33.333754 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:33.333711 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:33.397391 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:33.397310 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:35.621195 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:35.621156 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5rwf8" event={"ID":"48f171b4-a19f-4599-ab23-ce6dd9e968c0","Type":"ContainerStarted","Data":"65376729ce3f5afb44472c95a1ad615eee1374a17193c47c0ad2d56774d6823e"} Jul 31 23:02:35.637369 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:35.637307 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5rwf8" podStartSLOduration=1.422880802 podStartE2EDuration="4.637287801s" totalImagesPullingTime="3.214406999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:02:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:02:35.635317644 +0000 UTC m=+312.107761827" watchObservedRunningTime="2026-07-31 23:02:35.637287801 +0000 UTC m=+312.109731984" Jul 31 23:02:35.899698 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:35.899612 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:39.118820 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:39.118786 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:40.389976 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:40.389924 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:40.491284 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:40.491244 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:42.236843 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:42.236809 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88zcc"] Jul 31 23:02:42.240124 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:42.240105 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88zcc" Jul 31 23:02:42.242685 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:42.242665 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 31 23:02:42.243718 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:42.243704 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-r8dkg\"" Jul 31 23:02:42.243771 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:42.243707 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 31 23:02:42.249538 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:42.249494 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88zcc"] Jul 31 23:02:42.421800 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:42.421761 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k5xn\" (UniqueName: \"kubernetes.io/projected/70848628-570d-4c14-96bc-2163751b3bdb-kube-api-access-5k5xn\") pod \"openshift-lws-operator-fd8ccff4c-88zcc\" (UID: \"70848628-570d-4c14-96bc-2163751b3bdb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88zcc" Jul 31 23:02:42.421990 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:42.421821 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70848628-570d-4c14-96bc-2163751b3bdb-tmp\") pod \"openshift-lws-operator-fd8ccff4c-88zcc\" (UID: \"70848628-570d-4c14-96bc-2163751b3bdb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88zcc" Jul 31 23:02:42.523104 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:42.523017 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70848628-570d-4c14-96bc-2163751b3bdb-tmp\") pod \"openshift-lws-operator-fd8ccff4c-88zcc\" (UID: \"70848628-570d-4c14-96bc-2163751b3bdb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88zcc" Jul 31 23:02:42.523270 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:42.523123 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k5xn\" (UniqueName: \"kubernetes.io/projected/70848628-570d-4c14-96bc-2163751b3bdb-kube-api-access-5k5xn\") pod \"openshift-lws-operator-fd8ccff4c-88zcc\" (UID: \"70848628-570d-4c14-96bc-2163751b3bdb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88zcc" Jul 31 23:02:42.523437 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:42.523414 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70848628-570d-4c14-96bc-2163751b3bdb-tmp\") pod \"openshift-lws-operator-fd8ccff4c-88zcc\" (UID: \"70848628-570d-4c14-96bc-2163751b3bdb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88zcc" Jul 31 23:02:42.531848 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:42.531825 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k5xn\" (UniqueName: \"kubernetes.io/projected/70848628-570d-4c14-96bc-2163751b3bdb-kube-api-access-5k5xn\") pod \"openshift-lws-operator-fd8ccff4c-88zcc\" (UID: \"70848628-570d-4c14-96bc-2163751b3bdb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88zcc" Jul 31 23:02:42.549701 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:42.549674 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88zcc" Jul 31 23:02:42.681590 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:42.681446 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88zcc"] Jul 31 23:02:42.684395 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:02:42.684366 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70848628_570d_4c14_96bc_2163751b3bdb.slice/crio-304384eaabb79b38955b9df3961ef6b3400cafd2617f7c6b5a32c05aaf3c6ef7 WatchSource:0}: Error finding container 304384eaabb79b38955b9df3961ef6b3400cafd2617f7c6b5a32c05aaf3c6ef7: Status 404 returned error can't find the container with id 304384eaabb79b38955b9df3961ef6b3400cafd2617f7c6b5a32c05aaf3c6ef7 Jul 31 23:02:43.658518 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:43.658471 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88zcc" event={"ID":"70848628-570d-4c14-96bc-2163751b3bdb","Type":"ContainerStarted","Data":"304384eaabb79b38955b9df3961ef6b3400cafd2617f7c6b5a32c05aaf3c6ef7"} Jul 31 23:02:45.671209 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:45.671175 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88zcc" event={"ID":"70848628-570d-4c14-96bc-2163751b3bdb","Type":"ContainerStarted","Data":"d6b861691a876a28c5ed31514cdd3300345b6b722fc8eda65eb90964f3a84068"} Jul 31 23:02:45.687981 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:45.687935 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-88zcc" podStartSLOduration=0.911469606 podStartE2EDuration="3.687914491s" totalImagesPullingTime="2.776444885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:02:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:02:45.686121202 +0000 UTC m=+322.158565389" watchObservedRunningTime="2026-07-31 23:02:45.687914491 +0000 UTC m=+322.160358746" Jul 31 23:02:46.659095 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:46.659058 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:46.761242 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:46.761200 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:02:48.734798 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:02:48.734765 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:05.436112 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:05.436077 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:05.538695 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:05.538665 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:05.969735 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:05.969701 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:06.180853 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.180801 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:06.287414 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.287331 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:06.322454 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.322423 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9"] Jul 31 23:03:06.326227 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.326206 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.331256 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.331227 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 31 23:03:06.331544 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.331524 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 31 23:03:06.331652 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.331577 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 31 23:03:06.331652 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.331590 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 31 23:03:06.331863 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.331845 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zbcqc\"" Jul 31 23:03:06.353294 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.353258 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9"] Jul 31 23:03:06.428649 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.428615 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pmt9\" (UniqueName: \"kubernetes.io/projected/539eb9ef-f6bf-4161-a2fe-e246296d16d8-kube-api-access-9pmt9\") pod \"opendatahub-operator-controller-manager-8567dd98f6-9dfl9\" (UID: \"539eb9ef-f6bf-4161-a2fe-e246296d16d8\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.428823 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.428657 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/539eb9ef-f6bf-4161-a2fe-e246296d16d8-manifests\") pod \"opendatahub-operator-controller-manager-8567dd98f6-9dfl9\" (UID: \"539eb9ef-f6bf-4161-a2fe-e246296d16d8\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.428823 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.428685 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/539eb9ef-f6bf-4161-a2fe-e246296d16d8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8567dd98f6-9dfl9\" (UID: \"539eb9ef-f6bf-4161-a2fe-e246296d16d8\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.428823 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.428755 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/539eb9ef-f6bf-4161-a2fe-e246296d16d8-tmp\") pod \"opendatahub-operator-controller-manager-8567dd98f6-9dfl9\" (UID: \"539eb9ef-f6bf-4161-a2fe-e246296d16d8\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.428823 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.428806 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/539eb9ef-f6bf-4161-a2fe-e246296d16d8-webhook-cert\") pod \"opendatahub-operator-controller-manager-8567dd98f6-9dfl9\" (UID: \"539eb9ef-f6bf-4161-a2fe-e246296d16d8\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.435367 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.435335 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:06.529776 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.529743 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/539eb9ef-f6bf-4161-a2fe-e246296d16d8-tmp\") pod \"opendatahub-operator-controller-manager-8567dd98f6-9dfl9\" (UID: \"539eb9ef-f6bf-4161-a2fe-e246296d16d8\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.530160 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.529787 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/539eb9ef-f6bf-4161-a2fe-e246296d16d8-webhook-cert\") pod \"opendatahub-operator-controller-manager-8567dd98f6-9dfl9\" (UID: \"539eb9ef-f6bf-4161-a2fe-e246296d16d8\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.530160 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.529836 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pmt9\" (UniqueName: \"kubernetes.io/projected/539eb9ef-f6bf-4161-a2fe-e246296d16d8-kube-api-access-9pmt9\") pod \"opendatahub-operator-controller-manager-8567dd98f6-9dfl9\" (UID: \"539eb9ef-f6bf-4161-a2fe-e246296d16d8\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.530160 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.529859 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/539eb9ef-f6bf-4161-a2fe-e246296d16d8-manifests\") pod \"opendatahub-operator-controller-manager-8567dd98f6-9dfl9\" (UID: \"539eb9ef-f6bf-4161-a2fe-e246296d16d8\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.530160 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.529885 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/539eb9ef-f6bf-4161-a2fe-e246296d16d8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8567dd98f6-9dfl9\" (UID: \"539eb9ef-f6bf-4161-a2fe-e246296d16d8\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.530342 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.530263 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/539eb9ef-f6bf-4161-a2fe-e246296d16d8-manifests\") pod \"opendatahub-operator-controller-manager-8567dd98f6-9dfl9\" (UID: \"539eb9ef-f6bf-4161-a2fe-e246296d16d8\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.532052 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.532026 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/539eb9ef-f6bf-4161-a2fe-e246296d16d8-tmp\") pod \"opendatahub-operator-controller-manager-8567dd98f6-9dfl9\" (UID: \"539eb9ef-f6bf-4161-a2fe-e246296d16d8\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.532297 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.532280 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/539eb9ef-f6bf-4161-a2fe-e246296d16d8-webhook-cert\") pod \"opendatahub-operator-controller-manager-8567dd98f6-9dfl9\" (UID: \"539eb9ef-f6bf-4161-a2fe-e246296d16d8\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.532338 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.532324 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/539eb9ef-f6bf-4161-a2fe-e246296d16d8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8567dd98f6-9dfl9\" (UID: \"539eb9ef-f6bf-4161-a2fe-e246296d16d8\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.539005 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.538951 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pmt9\" (UniqueName: \"kubernetes.io/projected/539eb9ef-f6bf-4161-a2fe-e246296d16d8-kube-api-access-9pmt9\") pod \"opendatahub-operator-controller-manager-8567dd98f6-9dfl9\" (UID: \"539eb9ef-f6bf-4161-a2fe-e246296d16d8\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.545727 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.545676 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:06.636650 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.636611 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:06.645363 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.645336 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:06.745312 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.745281 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:06.778424 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.778400 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9"] Jul 31 23:03:06.780436 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:03:06.780411 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod539eb9ef_f6bf_4161_a2fe_e246296d16d8.slice/crio-bae2ade41c82521fd133468108c86b3364857eb9e6d9395b001adbe04127ba03 WatchSource:0}: Error finding container bae2ade41c82521fd133468108c86b3364857eb9e6d9395b001adbe04127ba03: Status 404 returned error can't find the container with id bae2ade41c82521fd133468108c86b3364857eb9e6d9395b001adbe04127ba03 Jul 31 23:03:06.847809 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:06.847780 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:07.765068 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:07.765029 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" event={"ID":"539eb9ef-f6bf-4161-a2fe-e246296d16d8","Type":"ContainerStarted","Data":"bae2ade41c82521fd133468108c86b3364857eb9e6d9395b001adbe04127ba03"} Jul 31 23:03:09.776046 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:09.776019 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" event={"ID":"539eb9ef-f6bf-4161-a2fe-e246296d16d8","Type":"ContainerStarted","Data":"46f848f4c21d52166d3248f23a528346260891c7905d4d98442d8e65730fab33"} Jul 31 23:03:11.786039 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:11.786002 2561 generic.go:356] "Generic (PLEG): container finished" podID="539eb9ef-f6bf-4161-a2fe-e246296d16d8" containerID="46f848f4c21d52166d3248f23a528346260891c7905d4d98442d8e65730fab33" exitCode=0 Jul 31 23:03:11.786434 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:11.786047 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" event={"ID":"539eb9ef-f6bf-4161-a2fe-e246296d16d8","Type":"ContainerDied","Data":"46f848f4c21d52166d3248f23a528346260891c7905d4d98442d8e65730fab33"} Jul 31 23:03:12.792700 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:12.792665 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" event={"ID":"539eb9ef-f6bf-4161-a2fe-e246296d16d8","Type":"ContainerStarted","Data":"310a7b4c83d64afb2718afb1db52b7c8fca6bf33963637df12ae8c43499095ef"} Jul 31 23:03:12.793120 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:12.792877 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:12.814355 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:12.814301 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" podStartSLOduration=2.444988774 podStartE2EDuration="6.814273079s" totalImagesPullingTime="3.2992329s" totalInitContainerRuntime="1.070051405s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:03:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:03:12.813363391 +0000 UTC m=+349.285807598" watchObservedRunningTime="2026-07-31 23:03:12.814273079 +0000 UTC m=+349.286717261" Jul 31 23:03:14.869431 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:14.869376 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:14.973856 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:14.973825 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:17.904872 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:17.904840 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:18.005828 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:18.005801 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:23.799126 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:23.799097 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-9dfl9" Jul 31 23:03:25.468090 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:25.468051 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:25.678834 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:25.678797 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:25.872564 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:25.872529 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:26.442856 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:26.442822 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:26.520342 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:26.520311 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:26.630374 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:26.630341 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:26.725230 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:26.725143 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:26.821475 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:26.821422 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:27.360911 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:27.360879 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:27.464280 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:27.464245 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:27.562149 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:27.562117 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:28.269368 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:28.269334 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:28.543589 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:28.543480 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:34.187721 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:34.187688 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:34.328827 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:34.328794 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:35.200854 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.200825 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7"] Jul 31 23:03:35.206600 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.206574 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.209455 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.209435 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 31 23:03:35.209791 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.209774 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-n7m97\"" Jul 31 23:03:35.210046 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.210025 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 31 23:03:35.210125 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.210065 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 31 23:03:35.227309 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.227271 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7"] Jul 31 23:03:35.246614 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.246577 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:35.290519 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.290480 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b19028-0f19-4880-ae03-e4654e10fc3d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.290707 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.290541 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86b19028-0f19-4880-ae03-e4654e10fc3d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.290707 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.290566 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86b19028-0f19-4880-ae03-e4654e10fc3d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.290707 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.290597 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86b19028-0f19-4880-ae03-e4654e10fc3d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.290707 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.290635 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86b19028-0f19-4880-ae03-e4654e10fc3d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.290707 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.290705 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkw8n\" (UniqueName: \"kubernetes.io/projected/86b19028-0f19-4880-ae03-e4654e10fc3d-kube-api-access-kkw8n\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.290905 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.290758 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b19028-0f19-4880-ae03-e4654e10fc3d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.290905 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.290777 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86b19028-0f19-4880-ae03-e4654e10fc3d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.290905 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.290826 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86b19028-0f19-4880-ae03-e4654e10fc3d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.392029 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.391992 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86b19028-0f19-4880-ae03-e4654e10fc3d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.392029 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.392030 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86b19028-0f19-4880-ae03-e4654e10fc3d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.392276 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.392206 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86b19028-0f19-4880-ae03-e4654e10fc3d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.392276 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.392231 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86b19028-0f19-4880-ae03-e4654e10fc3d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.392390 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.392280 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkw8n\" (UniqueName: \"kubernetes.io/projected/86b19028-0f19-4880-ae03-e4654e10fc3d-kube-api-access-kkw8n\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.392390 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.392353 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b19028-0f19-4880-ae03-e4654e10fc3d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.392390 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.392371 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86b19028-0f19-4880-ae03-e4654e10fc3d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.392574 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.392438 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86b19028-0f19-4880-ae03-e4654e10fc3d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.392574 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.392526 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86b19028-0f19-4880-ae03-e4654e10fc3d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.392692 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.392570 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b19028-0f19-4880-ae03-e4654e10fc3d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.392692 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.392641 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b19028-0f19-4880-ae03-e4654e10fc3d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.392856 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.392831 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86b19028-0f19-4880-ae03-e4654e10fc3d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.392992 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.392904 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b19028-0f19-4880-ae03-e4654e10fc3d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.392992 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.392946 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86b19028-0f19-4880-ae03-e4654e10fc3d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.394654 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.394632 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86b19028-0f19-4880-ae03-e4654e10fc3d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.394978 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.394953 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86b19028-0f19-4880-ae03-e4654e10fc3d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.415710 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.415670 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkw8n\" (UniqueName: \"kubernetes.io/projected/86b19028-0f19-4880-ae03-e4654e10fc3d-kube-api-access-kkw8n\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.416920 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.416900 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86b19028-0f19-4880-ae03-e4654e10fc3d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7\" (UID: \"86b19028-0f19-4880-ae03-e4654e10fc3d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.457255 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.457180 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:35.518253 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.518224 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:35.596753 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.596722 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:35.669374 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.669342 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7"] Jul 31 23:03:35.673321 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:03:35.673296 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86b19028_0f19_4880_ae03_e4654e10fc3d.slice/crio-12a8fcfd172dd89aacd972b22dea09149d2b12507d8ffa7c56f2d0a1ba167616 WatchSource:0}: Error finding container 12a8fcfd172dd89aacd972b22dea09149d2b12507d8ffa7c56f2d0a1ba167616: Status 404 returned error can't find the container with id 12a8fcfd172dd89aacd972b22dea09149d2b12507d8ffa7c56f2d0a1ba167616 Jul 31 23:03:35.896441 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:35.896407 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" event={"ID":"86b19028-0f19-4880-ae03-e4654e10fc3d","Type":"ContainerStarted","Data":"12a8fcfd172dd89aacd972b22dea09149d2b12507d8ffa7c56f2d0a1ba167616"} Jul 31 23:03:36.552267 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.552226 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp"] Jul 31 23:03:36.557777 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.557749 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:36.561760 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.561727 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 31 23:03:36.561902 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.561787 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 31 23:03:36.561902 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.561806 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 31 23:03:36.562022 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.561899 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kwgbf\"" Jul 31 23:03:36.576892 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.576862 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp"] Jul 31 23:03:36.604886 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.604852 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1930ccb7-2f6c-45d6-a910-c880719b32f3-cert\") pod \"lws-controller-manager-6b79896bff-bbcbp\" (UID: \"1930ccb7-2f6c-45d6-a910-c880719b32f3\") " pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:36.605072 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.604899 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1930ccb7-2f6c-45d6-a910-c880719b32f3-metrics-cert\") pod \"lws-controller-manager-6b79896bff-bbcbp\" (UID: \"1930ccb7-2f6c-45d6-a910-c880719b32f3\") " pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:36.605072 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.605006 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmkfd\" (UniqueName: \"kubernetes.io/projected/1930ccb7-2f6c-45d6-a910-c880719b32f3-kube-api-access-qmkfd\") pod \"lws-controller-manager-6b79896bff-bbcbp\" (UID: \"1930ccb7-2f6c-45d6-a910-c880719b32f3\") " pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:36.605214 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.605095 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1930ccb7-2f6c-45d6-a910-c880719b32f3-manager-config\") pod \"lws-controller-manager-6b79896bff-bbcbp\" (UID: \"1930ccb7-2f6c-45d6-a910-c880719b32f3\") " pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:36.706462 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.706425 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmkfd\" (UniqueName: \"kubernetes.io/projected/1930ccb7-2f6c-45d6-a910-c880719b32f3-kube-api-access-qmkfd\") pod \"lws-controller-manager-6b79896bff-bbcbp\" (UID: \"1930ccb7-2f6c-45d6-a910-c880719b32f3\") " pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:36.706723 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.706545 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1930ccb7-2f6c-45d6-a910-c880719b32f3-manager-config\") pod \"lws-controller-manager-6b79896bff-bbcbp\" (UID: \"1930ccb7-2f6c-45d6-a910-c880719b32f3\") " pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:36.706723 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.706619 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1930ccb7-2f6c-45d6-a910-c880719b32f3-cert\") pod \"lws-controller-manager-6b79896bff-bbcbp\" (UID: \"1930ccb7-2f6c-45d6-a910-c880719b32f3\") " pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:36.706723 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.706649 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1930ccb7-2f6c-45d6-a910-c880719b32f3-metrics-cert\") pod \"lws-controller-manager-6b79896bff-bbcbp\" (UID: \"1930ccb7-2f6c-45d6-a910-c880719b32f3\") " pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:36.707345 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.707273 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1930ccb7-2f6c-45d6-a910-c880719b32f3-manager-config\") pod \"lws-controller-manager-6b79896bff-bbcbp\" (UID: \"1930ccb7-2f6c-45d6-a910-c880719b32f3\") " pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:36.709556 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.709529 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1930ccb7-2f6c-45d6-a910-c880719b32f3-metrics-cert\") pod \"lws-controller-manager-6b79896bff-bbcbp\" (UID: \"1930ccb7-2f6c-45d6-a910-c880719b32f3\") " pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:36.710209 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.710186 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1930ccb7-2f6c-45d6-a910-c880719b32f3-cert\") pod \"lws-controller-manager-6b79896bff-bbcbp\" (UID: \"1930ccb7-2f6c-45d6-a910-c880719b32f3\") " pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:36.718989 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.718961 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmkfd\" (UniqueName: \"kubernetes.io/projected/1930ccb7-2f6c-45d6-a910-c880719b32f3-kube-api-access-qmkfd\") pod \"lws-controller-manager-6b79896bff-bbcbp\" (UID: \"1930ccb7-2f6c-45d6-a910-c880719b32f3\") " pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:36.873332 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:36.873283 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:37.027561 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:37.027530 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp"] Jul 31 23:03:37.030371 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:03:37.030342 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1930ccb7_2f6c_45d6_a910_c880719b32f3.slice/crio-3dc9043c4a4709ab50f2a053dbcdc41306d28fee43eb46b3a965b97f8af8cb82 WatchSource:0}: Error finding container 3dc9043c4a4709ab50f2a053dbcdc41306d28fee43eb46b3a965b97f8af8cb82: Status 404 returned error can't find the container with id 3dc9043c4a4709ab50f2a053dbcdc41306d28fee43eb46b3a965b97f8af8cb82 Jul 31 23:03:37.909304 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:37.909259 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" event={"ID":"1930ccb7-2f6c-45d6-a910-c880719b32f3","Type":"ContainerStarted","Data":"3dc9043c4a4709ab50f2a053dbcdc41306d28fee43eb46b3a965b97f8af8cb82"} Jul 31 23:03:37.937843 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:37.937809 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:38.037615 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.037576 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx"] Jul 31 23:03:38.044822 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.044789 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx" Jul 31 23:03:38.047673 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.047646 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 31 23:03:38.047673 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.047653 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jwfpg\"" Jul 31 23:03:38.047871 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.047832 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:38.054862 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.054833 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx"] Jul 31 23:03:38.120717 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.120677 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76d59ae5-39ee-42a1-896b-93c7a91c9db1-cert\") pod \"kserve-controller-manager-d6f4b65b8-4fhpx\" (UID: \"76d59ae5-39ee-42a1-896b-93c7a91c9db1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx" Jul 31 23:03:38.120948 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.120822 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbmq2\" (UniqueName: \"kubernetes.io/projected/76d59ae5-39ee-42a1-896b-93c7a91c9db1-kube-api-access-qbmq2\") pod \"kserve-controller-manager-d6f4b65b8-4fhpx\" (UID: \"76d59ae5-39ee-42a1-896b-93c7a91c9db1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx" Jul 31 23:03:38.221736 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.221647 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbmq2\" (UniqueName: \"kubernetes.io/projected/76d59ae5-39ee-42a1-896b-93c7a91c9db1-kube-api-access-qbmq2\") pod \"kserve-controller-manager-d6f4b65b8-4fhpx\" (UID: \"76d59ae5-39ee-42a1-896b-93c7a91c9db1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx" Jul 31 23:03:38.221909 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.221742 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76d59ae5-39ee-42a1-896b-93c7a91c9db1-cert\") pod \"kserve-controller-manager-d6f4b65b8-4fhpx\" (UID: \"76d59ae5-39ee-42a1-896b-93c7a91c9db1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx" Jul 31 23:03:38.222315 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:03:38.222284 2561 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 31 23:03:38.222402 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:03:38.222359 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76d59ae5-39ee-42a1-896b-93c7a91c9db1-cert podName:76d59ae5-39ee-42a1-896b-93c7a91c9db1 nodeName:}" failed. No retries permitted until 2026-07-31 23:03:38.722336732 +0000 UTC m=+375.194780893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/76d59ae5-39ee-42a1-896b-93c7a91c9db1-cert") pod "kserve-controller-manager-d6f4b65b8-4fhpx" (UID: "76d59ae5-39ee-42a1-896b-93c7a91c9db1") : secret "kserve-webhook-server-cert" not found Jul 31 23:03:38.249209 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.249172 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbmq2\" (UniqueName: \"kubernetes.io/projected/76d59ae5-39ee-42a1-896b-93c7a91c9db1-kube-api-access-qbmq2\") pod \"kserve-controller-manager-d6f4b65b8-4fhpx\" (UID: \"76d59ae5-39ee-42a1-896b-93c7a91c9db1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx" Jul 31 23:03:38.328374 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.328337 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:38.478606 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.478492 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:38.517462 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.517427 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:03:38.517611 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.517543 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:03:38.517611 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.517595 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:03:38.726778 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.726741 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76d59ae5-39ee-42a1-896b-93c7a91c9db1-cert\") pod \"kserve-controller-manager-d6f4b65b8-4fhpx\" (UID: \"76d59ae5-39ee-42a1-896b-93c7a91c9db1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx" Jul 31 23:03:38.730342 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.730244 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76d59ae5-39ee-42a1-896b-93c7a91c9db1-cert\") pod \"kserve-controller-manager-d6f4b65b8-4fhpx\" (UID: \"76d59ae5-39ee-42a1-896b-93c7a91c9db1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx" Jul 31 23:03:38.917374 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.917332 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" event={"ID":"86b19028-0f19-4880-ae03-e4654e10fc3d","Type":"ContainerStarted","Data":"45852c95aecb02182a0a95691f65baa2214d1f9123aa7d901e12362ba233cb54"} Jul 31 23:03:38.958272 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.958232 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx" Jul 31 23:03:38.965239 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:38.965179 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" podStartSLOduration=1.123252228 podStartE2EDuration="3.965154556s" totalImagesPullingTime="2.841902328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:03:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:03:38.963403085 +0000 UTC m=+375.435847283" watchObservedRunningTime="2026-07-31 23:03:38.965154556 +0000 UTC m=+375.437598741" Jul 31 23:03:39.138793 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:39.138759 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx"] Jul 31 23:03:39.234336 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:39.234289 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:39.332102 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:39.332067 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:39.384025 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:03:39.383993 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d59ae5_39ee_42a1_896b_93c7a91c9db1.slice/crio-b8c4b2108c4f92e75802ed84e19ba1dc12c080ffb4a7bd5e332c7eae15db5fe8 WatchSource:0}: Error finding container b8c4b2108c4f92e75802ed84e19ba1dc12c080ffb4a7bd5e332c7eae15db5fe8: Status 404 returned error can't find the container with id b8c4b2108c4f92e75802ed84e19ba1dc12c080ffb4a7bd5e332c7eae15db5fe8 Jul 31 23:03:39.441900 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:39.441862 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:39.519049 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:39.519027 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:39.523851 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:39.523825 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:39.923282 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:39.923236 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" event={"ID":"1930ccb7-2f6c-45d6-a910-c880719b32f3","Type":"ContainerStarted","Data":"a85b55f5df722399e8ad1d7117c08452465ad743531e09e47d2125b5992f9b28"} Jul 31 23:03:39.923792 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:39.923340 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:39.924444 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:39.924424 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx" event={"ID":"76d59ae5-39ee-42a1-896b-93c7a91c9db1","Type":"ContainerStarted","Data":"b8c4b2108c4f92e75802ed84e19ba1dc12c080ffb4a7bd5e332c7eae15db5fe8"} Jul 31 23:03:39.924640 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:39.924627 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:39.925567 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:39.925551 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7" Jul 31 23:03:39.940781 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:39.940736 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" podStartSLOduration=1.525614399 podStartE2EDuration="3.940722542s" totalImagesPullingTime="2.415108143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:03:39.939542803 +0000 UTC m=+376.411987006" watchObservedRunningTime="2026-07-31 23:03:39.940722542 +0000 UTC m=+376.413166725" Jul 31 23:03:42.943471 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:42.943429 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx" event={"ID":"76d59ae5-39ee-42a1-896b-93c7a91c9db1","Type":"ContainerStarted","Data":"b98671f817bdc49458a0b704b5a9beb0a70c478739e853cb329f51b7927cad9f"} Jul 31 23:03:42.943877 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:42.943551 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx" Jul 31 23:03:42.961598 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:42.961545 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx" podStartSLOduration=1.93065425 podStartE2EDuration="4.961529756s" totalImagesPullingTime="3.030875506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:03:42.961046946 +0000 UTC m=+379.433491142" watchObservedRunningTime="2026-07-31 23:03:42.961529756 +0000 UTC m=+379.433973938" Jul 31 23:03:50.934591 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:50.934555 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6b79896bff-bbcbp" Jul 31 23:03:54.234484 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:54.234452 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:54.291721 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:54.291681 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:03:58.476796 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:03:58.476758 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:05.654678 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:05.654640 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:05.755404 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:05.755369 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:05.822659 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:05.822631 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:13.953640 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:13.953611 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4fhpx" Jul 31 23:04:27.179056 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:27.179021 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:27.282355 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:27.282321 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:39.101492 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:39.101454 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:39.959105 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:39.959064 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:40.018810 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:40.018775 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:41.157636 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:41.157588 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm"] Jul 31 23:04:41.161773 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:41.161748 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm" Jul 31 23:04:41.166160 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:41.166130 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 31 23:04:41.166977 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:41.166950 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-5kqxp\"" Jul 31 23:04:41.166977 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:41.166978 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 31 23:04:41.175101 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:41.175072 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm"] Jul 31 23:04:41.306772 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:41.306731 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvmzn\" (UniqueName: \"kubernetes.io/projected/8aee4d38-c4e9-44a3-85a1-513328563c14-kube-api-access-vvmzn\") pod \"limitador-operator-controller-manager-6c549b6d85-2hqnm\" (UID: \"8aee4d38-c4e9-44a3-85a1-513328563c14\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm" Jul 31 23:04:41.407452 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:41.407415 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvmzn\" (UniqueName: \"kubernetes.io/projected/8aee4d38-c4e9-44a3-85a1-513328563c14-kube-api-access-vvmzn\") pod \"limitador-operator-controller-manager-6c549b6d85-2hqnm\" (UID: \"8aee4d38-c4e9-44a3-85a1-513328563c14\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm" Jul 31 23:04:41.426068 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:41.426001 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvmzn\" (UniqueName: \"kubernetes.io/projected/8aee4d38-c4e9-44a3-85a1-513328563c14-kube-api-access-vvmzn\") pod \"limitador-operator-controller-manager-6c549b6d85-2hqnm\" (UID: \"8aee4d38-c4e9-44a3-85a1-513328563c14\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm" Jul 31 23:04:41.475009 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:41.474976 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm" Jul 31 23:04:41.620519 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:41.620479 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm"] Jul 31 23:04:41.623032 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:04:41.622998 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aee4d38_c4e9_44a3_85a1_513328563c14.slice/crio-09b560445fa1aa096373988eb61dd5ac6f9507640d3b3b63eb0ff014c9a65991 WatchSource:0}: Error finding container 09b560445fa1aa096373988eb61dd5ac6f9507640d3b3b63eb0ff014c9a65991: Status 404 returned error can't find the container with id 09b560445fa1aa096373988eb61dd5ac6f9507640d3b3b63eb0ff014c9a65991 Jul 31 23:04:41.915922 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:41.915890 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:42.214215 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:42.214130 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm" event={"ID":"8aee4d38-c4e9-44a3-85a1-513328563c14","Type":"ContainerStarted","Data":"09b560445fa1aa096373988eb61dd5ac6f9507640d3b3b63eb0ff014c9a65991"} Jul 31 23:04:43.565036 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:43.564997 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:43.940040 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:43.939931 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:44.228799 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:44.228702 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm" event={"ID":"8aee4d38-c4e9-44a3-85a1-513328563c14","Type":"ContainerStarted","Data":"598bf59e74f735fc33f3f3e7f6360cac6f8a0be0832f9a6d19f9acd4bd777a1f"} Jul 31 23:04:44.228969 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:44.228818 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm" Jul 31 23:04:44.263113 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:44.263036 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm" podStartSLOduration=0.840157104 podStartE2EDuration="3.263014402s" totalImagesPullingTime="2.422857298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:04:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:04:44.261044519 +0000 UTC m=+440.733488701" watchObservedRunningTime="2026-07-31 23:04:44.263014402 +0000 UTC m=+440.735458586" Jul 31 23:04:44.263399 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:44.263133 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:45.555482 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:45.555395 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:46.506002 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:46.505945 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:46.908673 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:46.908629 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:47.374234 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:47.374163 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:47.577385 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:47.577356 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:47.667519 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:47.667426 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:49.168355 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:49.168316 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:49.268559 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:49.268491 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:50.345060 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:50.345023 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd"] Jul 31 23:04:50.348895 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:50.348872 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:04:50.351906 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:50.351886 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mcgdw\"" Jul 31 23:04:50.365675 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:50.365645 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd"] Jul 31 23:04:50.494518 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:50.494461 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twb4w\" (UniqueName: \"kubernetes.io/projected/35fcfff2-4e62-41db-b545-519d5d2a8d60-kube-api-access-twb4w\") pod \"kuadrant-operator-controller-manager-58fc56757c-wfqxd\" (UID: \"35fcfff2-4e62-41db-b545-519d5d2a8d60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:04:50.494695 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:50.494603 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35fcfff2-4e62-41db-b545-519d5d2a8d60-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-wfqxd\" (UID: \"35fcfff2-4e62-41db-b545-519d5d2a8d60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:04:50.595696 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:50.595603 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twb4w\" (UniqueName: \"kubernetes.io/projected/35fcfff2-4e62-41db-b545-519d5d2a8d60-kube-api-access-twb4w\") pod \"kuadrant-operator-controller-manager-58fc56757c-wfqxd\" (UID: \"35fcfff2-4e62-41db-b545-519d5d2a8d60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:04:50.595696 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:50.595688 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35fcfff2-4e62-41db-b545-519d5d2a8d60-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-wfqxd\" (UID: \"35fcfff2-4e62-41db-b545-519d5d2a8d60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:04:50.596000 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:50.595984 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35fcfff2-4e62-41db-b545-519d5d2a8d60-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-wfqxd\" (UID: \"35fcfff2-4e62-41db-b545-519d5d2a8d60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:04:50.604988 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:50.604956 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twb4w\" (UniqueName: \"kubernetes.io/projected/35fcfff2-4e62-41db-b545-519d5d2a8d60-kube-api-access-twb4w\") pod \"kuadrant-operator-controller-manager-58fc56757c-wfqxd\" (UID: \"35fcfff2-4e62-41db-b545-519d5d2a8d60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:04:50.663853 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:50.663809 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:04:50.805687 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:50.805657 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd"] Jul 31 23:04:50.808376 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:04:50.808349 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35fcfff2_4e62_41db_b545_519d5d2a8d60.slice/crio-41d56547c9428ca3eb9cebf91849be4a1246151f31de869bab1549c8f4eee2f5 WatchSource:0}: Error finding container 41d56547c9428ca3eb9cebf91849be4a1246151f31de869bab1549c8f4eee2f5: Status 404 returned error can't find the container with id 41d56547c9428ca3eb9cebf91849be4a1246151f31de869bab1549c8f4eee2f5 Jul 31 23:04:51.262008 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:51.261977 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" event={"ID":"35fcfff2-4e62-41db-b545-519d5d2a8d60","Type":"ContainerStarted","Data":"41d56547c9428ca3eb9cebf91849be4a1246151f31de869bab1549c8f4eee2f5"} Jul 31 23:04:51.706129 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:51.706099 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:51.806219 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:51.806181 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:54.737733 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:54.737697 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:54.838374 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:54.838339 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:55.235898 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:55.235656 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm" Jul 31 23:04:55.288528 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:55.288461 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" event={"ID":"35fcfff2-4e62-41db-b545-519d5d2a8d60","Type":"ContainerStarted","Data":"d9875393cea3893caef2b6e32b1fe92e4718850938ed1c126a2931adf96eda10"} Jul 31 23:04:55.288730 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:55.288569 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:04:55.308649 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:55.308592 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" podStartSLOduration=1.364723987 podStartE2EDuration="5.308570515s" totalImagesPullingTime="3.943846528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:04:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:04:55.306549648 +0000 UTC m=+451.778993827" watchObservedRunningTime="2026-07-31 23:04:55.308570515 +0000 UTC m=+451.781014697" Jul 31 23:04:57.197779 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:57.197738 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:04:57.403276 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:04:57.403231 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:00.348153 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:00.348119 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:00.451321 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:00.451286 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:05.098198 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:05.098155 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:05.196985 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:05.196941 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:06.295645 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:06.295618 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:05:07.969052 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:07.969011 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd"] Jul 31 23:05:07.969459 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:07.969339 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" podUID="35fcfff2-4e62-41db-b545-519d5d2a8d60" containerName="manager" containerID="cri-o://d9875393cea3893caef2b6e32b1fe92e4718850938ed1c126a2931adf96eda10" gracePeriod=2 Jul 31 23:05:07.984785 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:07.984755 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd"] Jul 31 23:05:07.990081 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:07.990033 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m"] Jul 31 23:05:07.990528 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:07.990449 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35fcfff2-4e62-41db-b545-519d5d2a8d60" containerName="manager" Jul 31 23:05:07.990528 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:07.990463 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="35fcfff2-4e62-41db-b545-519d5d2a8d60" containerName="manager" Jul 31 23:05:07.990645 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:07.990532 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="35fcfff2-4e62-41db-b545-519d5d2a8d60" containerName="manager" Jul 31 23:05:07.990945 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:05:07.990906 2561 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"35fcfff2-4e62-41db-b545-519d5d2a8d60\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-wfqxd\": pods \"kuadrant-operator-controller-manager-58fc56757c-wfqxd\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:05:07.994891 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:07.994873 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" Jul 31 23:05:07.997640 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:05:07.997605 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-wfqxd\" is forbidden: User \"system:node:ip-10-0-131-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-148.ec2.internal' and this object" podUID="35fcfff2-4e62-41db-b545-519d5d2a8d60" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:05:08.007967 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.007935 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m"] Jul 31 23:05:08.021514 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.021464 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm"] Jul 31 23:05:08.021737 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.021713 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm" podUID="8aee4d38-c4e9-44a3-85a1-513328563c14" containerName="manager" containerID="cri-o://598bf59e74f735fc33f3f3e7f6360cac6f8a0be0832f9a6d19f9acd4bd777a1f" gracePeriod=2 Jul 31 23:05:08.040063 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.040027 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm"] Jul 31 23:05:08.051215 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.051185 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qltht"] Jul 31 23:05:08.051755 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.051739 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aee4d38-c4e9-44a3-85a1-513328563c14" containerName="manager" Jul 31 23:05:08.051804 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.051760 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aee4d38-c4e9-44a3-85a1-513328563c14" containerName="manager" Jul 31 23:05:08.051886 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.051874 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aee4d38-c4e9-44a3-85a1-513328563c14" containerName="manager" Jul 31 23:05:08.053894 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:05:08.053845 2561 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"8aee4d38-c4e9-44a3-85a1-513328563c14\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-6c549b6d85-2hqnm\": pods \"limitador-operator-controller-manager-6c549b6d85-2hqnm\" not found" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm" Jul 31 23:05:08.055669 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.055650 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qltht" Jul 31 23:05:08.056078 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:05:08.056048 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-wfqxd\" is forbidden: User \"system:node:ip-10-0-131-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-148.ec2.internal' and this object" podUID="35fcfff2-4e62-41db-b545-519d5d2a8d60" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:05:08.057433 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.057409 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c41a9f04-4e30-49a4-b1ae-2424e890d562-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-bqw2m\" (UID: \"c41a9f04-4e30-49a4-b1ae-2424e890d562\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" Jul 31 23:05:08.057544 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.057509 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blngq\" (UniqueName: \"kubernetes.io/projected/c41a9f04-4e30-49a4-b1ae-2424e890d562-kube-api-access-blngq\") pod \"kuadrant-operator-controller-manager-58fc56757c-bqw2m\" (UID: \"c41a9f04-4e30-49a4-b1ae-2424e890d562\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" Jul 31 23:05:08.058050 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:05:08.058021 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-2hqnm\" is forbidden: User \"system:node:ip-10-0-131-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-148.ec2.internal' and this object" podUID="8aee4d38-c4e9-44a3-85a1-513328563c14" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm" Jul 31 23:05:08.060062 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:05:08.060035 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-wfqxd\" is forbidden: User \"system:node:ip-10-0-131-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-148.ec2.internal' and this object" podUID="35fcfff2-4e62-41db-b545-519d5d2a8d60" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:05:08.071305 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.071283 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qltht"] Jul 31 23:05:08.158395 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.158361 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfmd9\" (UniqueName: \"kubernetes.io/projected/7966c511-43a9-4a69-bd4e-e0b731ca092f-kube-api-access-jfmd9\") pod \"limitador-operator-controller-manager-6c549b6d85-qltht\" (UID: \"7966c511-43a9-4a69-bd4e-e0b731ca092f\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qltht" Jul 31 23:05:08.158603 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.158544 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blngq\" (UniqueName: \"kubernetes.io/projected/c41a9f04-4e30-49a4-b1ae-2424e890d562-kube-api-access-blngq\") pod \"kuadrant-operator-controller-manager-58fc56757c-bqw2m\" (UID: \"c41a9f04-4e30-49a4-b1ae-2424e890d562\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" Jul 31 23:05:08.158803 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.158776 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c41a9f04-4e30-49a4-b1ae-2424e890d562-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-bqw2m\" (UID: \"c41a9f04-4e30-49a4-b1ae-2424e890d562\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" Jul 31 23:05:08.159236 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.159210 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c41a9f04-4e30-49a4-b1ae-2424e890d562-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-bqw2m\" (UID: \"c41a9f04-4e30-49a4-b1ae-2424e890d562\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" Jul 31 23:05:08.168573 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.168534 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blngq\" (UniqueName: \"kubernetes.io/projected/c41a9f04-4e30-49a4-b1ae-2424e890d562-kube-api-access-blngq\") pod \"kuadrant-operator-controller-manager-58fc56757c-bqw2m\" (UID: \"c41a9f04-4e30-49a4-b1ae-2424e890d562\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" Jul 31 23:05:08.259424 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.259395 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfmd9\" (UniqueName: \"kubernetes.io/projected/7966c511-43a9-4a69-bd4e-e0b731ca092f-kube-api-access-jfmd9\") pod \"limitador-operator-controller-manager-6c549b6d85-qltht\" (UID: \"7966c511-43a9-4a69-bd4e-e0b731ca092f\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qltht" Jul 31 23:05:08.268183 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.268157 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:05:08.271100 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.271072 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm" Jul 31 23:05:08.273392 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.273368 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfmd9\" (UniqueName: \"kubernetes.io/projected/7966c511-43a9-4a69-bd4e-e0b731ca092f-kube-api-access-jfmd9\") pod \"limitador-operator-controller-manager-6c549b6d85-qltht\" (UID: \"7966c511-43a9-4a69-bd4e-e0b731ca092f\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qltht" Jul 31 23:05:08.353422 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.353388 2561 generic.go:356] "Generic (PLEG): container finished" podID="35fcfff2-4e62-41db-b545-519d5d2a8d60" containerID="d9875393cea3893caef2b6e32b1fe92e4718850938ed1c126a2931adf96eda10" exitCode=0 Jul 31 23:05:08.353422 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.353427 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-wfqxd" Jul 31 23:05:08.353683 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.353491 2561 scope.go:117] "RemoveContainer" containerID="d9875393cea3893caef2b6e32b1fe92e4718850938ed1c126a2931adf96eda10" Jul 31 23:05:08.354863 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.354842 2561 generic.go:356] "Generic (PLEG): container finished" podID="8aee4d38-c4e9-44a3-85a1-513328563c14" containerID="598bf59e74f735fc33f3f3e7f6360cac6f8a0be0832f9a6d19f9acd4bd777a1f" exitCode=0 Jul 31 23:05:08.354969 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.354880 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-2hqnm" Jul 31 23:05:08.360638 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.360609 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvmzn\" (UniqueName: \"kubernetes.io/projected/8aee4d38-c4e9-44a3-85a1-513328563c14-kube-api-access-vvmzn\") pod \"8aee4d38-c4e9-44a3-85a1-513328563c14\" (UID: \"8aee4d38-c4e9-44a3-85a1-513328563c14\") " Jul 31 23:05:08.360750 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.360647 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35fcfff2-4e62-41db-b545-519d5d2a8d60-extensions-socket-volume\") pod \"35fcfff2-4e62-41db-b545-519d5d2a8d60\" (UID: \"35fcfff2-4e62-41db-b545-519d5d2a8d60\") " Jul 31 23:05:08.360750 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.360679 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twb4w\" (UniqueName: \"kubernetes.io/projected/35fcfff2-4e62-41db-b545-519d5d2a8d60-kube-api-access-twb4w\") pod \"35fcfff2-4e62-41db-b545-519d5d2a8d60\" (UID: \"35fcfff2-4e62-41db-b545-519d5d2a8d60\") " Jul 31 23:05:08.361212 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.361190 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35fcfff2-4e62-41db-b545-519d5d2a8d60-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "35fcfff2-4e62-41db-b545-519d5d2a8d60" (UID: "35fcfff2-4e62-41db-b545-519d5d2a8d60"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:05:08.362093 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.362070 2561 scope.go:117] "RemoveContainer" containerID="d9875393cea3893caef2b6e32b1fe92e4718850938ed1c126a2931adf96eda10" Jul 31 23:05:08.362393 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:05:08.362375 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9875393cea3893caef2b6e32b1fe92e4718850938ed1c126a2931adf96eda10\": container with ID starting with d9875393cea3893caef2b6e32b1fe92e4718850938ed1c126a2931adf96eda10 not found: ID does not exist" containerID="d9875393cea3893caef2b6e32b1fe92e4718850938ed1c126a2931adf96eda10" Jul 31 23:05:08.362481 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.362406 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9875393cea3893caef2b6e32b1fe92e4718850938ed1c126a2931adf96eda10"} err="failed to get container status \"d9875393cea3893caef2b6e32b1fe92e4718850938ed1c126a2931adf96eda10\": rpc error: code = NotFound desc = could not find container \"d9875393cea3893caef2b6e32b1fe92e4718850938ed1c126a2931adf96eda10\": container with ID starting with d9875393cea3893caef2b6e32b1fe92e4718850938ed1c126a2931adf96eda10 not found: ID does not exist" Jul 31 23:05:08.362481 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.362443 2561 scope.go:117] "RemoveContainer" containerID="598bf59e74f735fc33f3f3e7f6360cac6f8a0be0832f9a6d19f9acd4bd777a1f" Jul 31 23:05:08.363092 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.363072 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35fcfff2-4e62-41db-b545-519d5d2a8d60-kube-api-access-twb4w" (OuterVolumeSpecName: "kube-api-access-twb4w") pod "35fcfff2-4e62-41db-b545-519d5d2a8d60" (UID: "35fcfff2-4e62-41db-b545-519d5d2a8d60"). InnerVolumeSpecName "kube-api-access-twb4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:05:08.363157 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.363076 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aee4d38-c4e9-44a3-85a1-513328563c14-kube-api-access-vvmzn" (OuterVolumeSpecName: "kube-api-access-vvmzn") pod "8aee4d38-c4e9-44a3-85a1-513328563c14" (UID: "8aee4d38-c4e9-44a3-85a1-513328563c14"). InnerVolumeSpecName "kube-api-access-vvmzn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:05:08.370246 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.370226 2561 scope.go:117] "RemoveContainer" containerID="598bf59e74f735fc33f3f3e7f6360cac6f8a0be0832f9a6d19f9acd4bd777a1f" Jul 31 23:05:08.370534 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:05:08.370488 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"598bf59e74f735fc33f3f3e7f6360cac6f8a0be0832f9a6d19f9acd4bd777a1f\": container with ID starting with 598bf59e74f735fc33f3f3e7f6360cac6f8a0be0832f9a6d19f9acd4bd777a1f not found: ID does not exist" containerID="598bf59e74f735fc33f3f3e7f6360cac6f8a0be0832f9a6d19f9acd4bd777a1f" Jul 31 23:05:08.370585 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.370546 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"598bf59e74f735fc33f3f3e7f6360cac6f8a0be0832f9a6d19f9acd4bd777a1f"} err="failed to get container status \"598bf59e74f735fc33f3f3e7f6360cac6f8a0be0832f9a6d19f9acd4bd777a1f\": rpc error: code = NotFound desc = could not find container \"598bf59e74f735fc33f3f3e7f6360cac6f8a0be0832f9a6d19f9acd4bd777a1f\": container with ID starting with 598bf59e74f735fc33f3f3e7f6360cac6f8a0be0832f9a6d19f9acd4bd777a1f not found: ID does not exist" Jul 31 23:05:08.420689 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.420651 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" Jul 31 23:05:08.426216 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.426195 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qltht" Jul 31 23:05:08.462007 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.461962 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvmzn\" (UniqueName: \"kubernetes.io/projected/8aee4d38-c4e9-44a3-85a1-513328563c14-kube-api-access-vvmzn\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:05:08.462007 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.462000 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35fcfff2-4e62-41db-b545-519d5d2a8d60-extensions-socket-volume\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:05:08.462007 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.462012 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twb4w\" (UniqueName: \"kubernetes.io/projected/35fcfff2-4e62-41db-b545-519d5d2a8d60-kube-api-access-twb4w\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:05:08.585350 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.585322 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m"] Jul 31 23:05:08.588322 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:05:08.588287 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc41a9f04_4e30_49a4_b1ae_2424e890d562.slice/crio-239ae2289e2a4b59f4731e8d5bb85febcf6d5cc327ecc7d5b1c0a33d4ed01b18 WatchSource:0}: Error finding container 239ae2289e2a4b59f4731e8d5bb85febcf6d5cc327ecc7d5b1c0a33d4ed01b18: Status 404 returned error can't find the container with id 239ae2289e2a4b59f4731e8d5bb85febcf6d5cc327ecc7d5b1c0a33d4ed01b18 Jul 31 23:05:08.591620 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:08.591591 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qltht"] Jul 31 23:05:08.594739 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:05:08.594715 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7966c511_43a9_4a69_bd4e_e0b731ca092f.slice/crio-b8aa7ae7c5c1109dcd7057be75e4ac4102afc3babb1184f28cf60022aadc2675 WatchSource:0}: Error finding container b8aa7ae7c5c1109dcd7057be75e4ac4102afc3babb1184f28cf60022aadc2675: Status 404 returned error can't find the container with id b8aa7ae7c5c1109dcd7057be75e4ac4102afc3babb1184f28cf60022aadc2675 Jul 31 23:05:09.364134 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:09.364089 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" event={"ID":"c41a9f04-4e30-49a4-b1ae-2424e890d562","Type":"ContainerStarted","Data":"9d259b57bd81a18d1fbdb335e850379b09df2f986cfe0709ff4361a6b5656938"} Jul 31 23:05:09.364637 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:09.364146 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" event={"ID":"c41a9f04-4e30-49a4-b1ae-2424e890d562","Type":"ContainerStarted","Data":"239ae2289e2a4b59f4731e8d5bb85febcf6d5cc327ecc7d5b1c0a33d4ed01b18"} Jul 31 23:05:09.364637 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:09.364203 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" Jul 31 23:05:09.365708 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:09.365686 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qltht" event={"ID":"7966c511-43a9-4a69-bd4e-e0b731ca092f","Type":"ContainerStarted","Data":"7febf236838a000cd16697812accb181935266fc387ca32e9fe07816c6b50ab1"} Jul 31 23:05:09.365807 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:09.365723 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qltht" event={"ID":"7966c511-43a9-4a69-bd4e-e0b731ca092f","Type":"ContainerStarted","Data":"b8aa7ae7c5c1109dcd7057be75e4ac4102afc3babb1184f28cf60022aadc2675"} Jul 31 23:05:09.365807 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:09.365766 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qltht" Jul 31 23:05:09.436184 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:09.436111 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" podStartSLOduration=2.436095881 podStartE2EDuration="2.436095881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:05:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:05:09.43461331 +0000 UTC m=+465.907057517" watchObservedRunningTime="2026-07-31 23:05:09.436095881 +0000 UTC m=+465.908540063" Jul 31 23:05:09.464263 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:09.464208 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qltht" podStartSLOduration=1.46418831 podStartE2EDuration="1.46418831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:05:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:05:09.46228602 +0000 UTC m=+465.934730202" watchObservedRunningTime="2026-07-31 23:05:09.46418831 +0000 UTC m=+465.936632493" Jul 31 23:05:09.913683 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:09.913651 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:10.132988 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:10.132949 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35fcfff2-4e62-41db-b545-519d5d2a8d60" path="/var/lib/kubelet/pods/35fcfff2-4e62-41db-b545-519d5d2a8d60/volumes" Jul 31 23:05:10.133303 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:10.133290 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8aee4d38-c4e9-44a3-85a1-513328563c14" path="/var/lib/kubelet/pods/8aee4d38-c4e9-44a3-85a1-513328563c14/volumes" Jul 31 23:05:11.810211 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:11.810179 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:11.908046 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:11.908013 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:14.721368 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:14.721335 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:14.822516 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:14.822469 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:20.373070 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:20.373034 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qltht" Jul 31 23:05:20.373449 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:20.373116 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" Jul 31 23:05:20.535135 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:20.535097 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:20.635275 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:20.635177 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:21.187739 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:21.187706 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:21.279317 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:21.279286 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:23.057176 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.057143 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m"] Jul 31 23:05:23.057651 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.057367 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" podUID="c41a9f04-4e30-49a4-b1ae-2424e890d562" containerName="manager" containerID="cri-o://9d259b57bd81a18d1fbdb335e850379b09df2f986cfe0709ff4361a6b5656938" gracePeriod=10 Jul 31 23:05:23.296697 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.296671 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" Jul 31 23:05:23.352364 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.352269 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg"] Jul 31 23:05:23.352760 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.352743 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c41a9f04-4e30-49a4-b1ae-2424e890d562" containerName="manager" Jul 31 23:05:23.352840 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.352763 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c41a9f04-4e30-49a4-b1ae-2424e890d562" containerName="manager" Jul 31 23:05:23.352900 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.352849 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c41a9f04-4e30-49a4-b1ae-2424e890d562" containerName="manager" Jul 31 23:05:23.354716 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.354695 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" Jul 31 23:05:23.370336 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.370302 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg"] Jul 31 23:05:23.406371 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.406339 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c41a9f04-4e30-49a4-b1ae-2424e890d562-extensions-socket-volume\") pod \"c41a9f04-4e30-49a4-b1ae-2424e890d562\" (UID: \"c41a9f04-4e30-49a4-b1ae-2424e890d562\") " Jul 31 23:05:23.406574 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.406386 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blngq\" (UniqueName: \"kubernetes.io/projected/c41a9f04-4e30-49a4-b1ae-2424e890d562-kube-api-access-blngq\") pod \"c41a9f04-4e30-49a4-b1ae-2424e890d562\" (UID: \"c41a9f04-4e30-49a4-b1ae-2424e890d562\") " Jul 31 23:05:23.406767 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.406744 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c41a9f04-4e30-49a4-b1ae-2424e890d562-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c41a9f04-4e30-49a4-b1ae-2424e890d562" (UID: "c41a9f04-4e30-49a4-b1ae-2424e890d562"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:05:23.408476 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.408452 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c41a9f04-4e30-49a4-b1ae-2424e890d562-kube-api-access-blngq" (OuterVolumeSpecName: "kube-api-access-blngq") pod "c41a9f04-4e30-49a4-b1ae-2424e890d562" (UID: "c41a9f04-4e30-49a4-b1ae-2424e890d562"). InnerVolumeSpecName "kube-api-access-blngq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:05:23.435353 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.435318 2561 generic.go:356] "Generic (PLEG): container finished" podID="c41a9f04-4e30-49a4-b1ae-2424e890d562" containerID="9d259b57bd81a18d1fbdb335e850379b09df2f986cfe0709ff4361a6b5656938" exitCode=0 Jul 31 23:05:23.435526 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.435363 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" Jul 31 23:05:23.435526 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.435396 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" event={"ID":"c41a9f04-4e30-49a4-b1ae-2424e890d562","Type":"ContainerDied","Data":"9d259b57bd81a18d1fbdb335e850379b09df2f986cfe0709ff4361a6b5656938"} Jul 31 23:05:23.435526 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.435465 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m" event={"ID":"c41a9f04-4e30-49a4-b1ae-2424e890d562","Type":"ContainerDied","Data":"239ae2289e2a4b59f4731e8d5bb85febcf6d5cc327ecc7d5b1c0a33d4ed01b18"} Jul 31 23:05:23.435526 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.435494 2561 scope.go:117] "RemoveContainer" containerID="9d259b57bd81a18d1fbdb335e850379b09df2f986cfe0709ff4361a6b5656938" Jul 31 23:05:23.444961 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.444945 2561 scope.go:117] "RemoveContainer" containerID="9d259b57bd81a18d1fbdb335e850379b09df2f986cfe0709ff4361a6b5656938" Jul 31 23:05:23.445231 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:05:23.445210 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d259b57bd81a18d1fbdb335e850379b09df2f986cfe0709ff4361a6b5656938\": container with ID starting with 9d259b57bd81a18d1fbdb335e850379b09df2f986cfe0709ff4361a6b5656938 not found: ID does not exist" containerID="9d259b57bd81a18d1fbdb335e850379b09df2f986cfe0709ff4361a6b5656938" Jul 31 23:05:23.445293 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.445240 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d259b57bd81a18d1fbdb335e850379b09df2f986cfe0709ff4361a6b5656938"} err="failed to get container status \"9d259b57bd81a18d1fbdb335e850379b09df2f986cfe0709ff4361a6b5656938\": rpc error: code = NotFound desc = could not find container \"9d259b57bd81a18d1fbdb335e850379b09df2f986cfe0709ff4361a6b5656938\": container with ID starting with 9d259b57bd81a18d1fbdb335e850379b09df2f986cfe0709ff4361a6b5656938 not found: ID does not exist" Jul 31 23:05:23.459989 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.459957 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m"] Jul 31 23:05:23.466013 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.465973 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-bqw2m"] Jul 31 23:05:23.507656 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.507615 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpbt2\" (UniqueName: \"kubernetes.io/projected/6b72cf24-2d85-4828-bdb5-16edeb5a8879-kube-api-access-zpbt2\") pod \"kuadrant-operator-controller-manager-68cf589589-55spg\" (UID: \"6b72cf24-2d85-4828-bdb5-16edeb5a8879\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" Jul 31 23:05:23.507837 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.507670 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b72cf24-2d85-4828-bdb5-16edeb5a8879-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-55spg\" (UID: \"6b72cf24-2d85-4828-bdb5-16edeb5a8879\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" Jul 31 23:05:23.507837 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.507773 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c41a9f04-4e30-49a4-b1ae-2424e890d562-extensions-socket-volume\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:05:23.507837 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.507792 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blngq\" (UniqueName: \"kubernetes.io/projected/c41a9f04-4e30-49a4-b1ae-2424e890d562-kube-api-access-blngq\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:05:23.608456 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.608362 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpbt2\" (UniqueName: \"kubernetes.io/projected/6b72cf24-2d85-4828-bdb5-16edeb5a8879-kube-api-access-zpbt2\") pod \"kuadrant-operator-controller-manager-68cf589589-55spg\" (UID: \"6b72cf24-2d85-4828-bdb5-16edeb5a8879\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" Jul 31 23:05:23.608456 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.608418 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b72cf24-2d85-4828-bdb5-16edeb5a8879-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-55spg\" (UID: \"6b72cf24-2d85-4828-bdb5-16edeb5a8879\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" Jul 31 23:05:23.608871 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.608851 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b72cf24-2d85-4828-bdb5-16edeb5a8879-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-55spg\" (UID: \"6b72cf24-2d85-4828-bdb5-16edeb5a8879\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" Jul 31 23:05:23.618133 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.618106 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpbt2\" (UniqueName: \"kubernetes.io/projected/6b72cf24-2d85-4828-bdb5-16edeb5a8879-kube-api-access-zpbt2\") pod \"kuadrant-operator-controller-manager-68cf589589-55spg\" (UID: \"6b72cf24-2d85-4828-bdb5-16edeb5a8879\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" Jul 31 23:05:23.665289 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.665248 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" Jul 31 23:05:23.811030 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:23.810995 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg"] Jul 31 23:05:23.815637 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:05:23.815608 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b72cf24_2d85_4828_bdb5_16edeb5a8879.slice/crio-7c1192a1182e6e0f6b63e802d3f99660bb97882b6cfc42b5688f29f933b8c7ab WatchSource:0}: Error finding container 7c1192a1182e6e0f6b63e802d3f99660bb97882b6cfc42b5688f29f933b8c7ab: Status 404 returned error can't find the container with id 7c1192a1182e6e0f6b63e802d3f99660bb97882b6cfc42b5688f29f933b8c7ab Jul 31 23:05:24.133468 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:24.133387 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c41a9f04-4e30-49a4-b1ae-2424e890d562" path="/var/lib/kubelet/pods/c41a9f04-4e30-49a4-b1ae-2424e890d562/volumes" Jul 31 23:05:24.443317 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:24.443213 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" event={"ID":"6b72cf24-2d85-4828-bdb5-16edeb5a8879","Type":"ContainerStarted","Data":"0758a13db3029b16bb7729832c34ec33aa77c8bc69d714aae62066654c54aade"} Jul 31 23:05:24.443317 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:24.443270 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" event={"ID":"6b72cf24-2d85-4828-bdb5-16edeb5a8879","Type":"ContainerStarted","Data":"7c1192a1182e6e0f6b63e802d3f99660bb97882b6cfc42b5688f29f933b8c7ab"} Jul 31 23:05:24.443542 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:24.443315 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" Jul 31 23:05:24.466893 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:24.466848 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" podStartSLOduration=1.466833135 podStartE2EDuration="1.466833135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:05:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:05:24.465445126 +0000 UTC m=+480.937889308" watchObservedRunningTime="2026-07-31 23:05:24.466833135 +0000 UTC m=+480.939277318" Jul 31 23:05:24.599582 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:24.599551 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:24.695266 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:24.695169 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:24.797095 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:24.797061 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:35.452207 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:35.452172 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" Jul 31 23:05:39.318205 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.318172 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n"] Jul 31 23:05:39.321770 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.321745 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.324413 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.324391 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-jgndk\"" Jul 31 23:05:39.333565 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.333495 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n"] Jul 31 23:05:39.341226 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.341180 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:39.349585 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.349548 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea5dcba5-3cc5-4452-a308-4c70c86762ce-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.349719 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.349591 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea5dcba5-3cc5-4452-a308-4c70c86762ce-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.349719 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.349660 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea5dcba5-3cc5-4452-a308-4c70c86762ce-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.349719 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.349692 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea5dcba5-3cc5-4452-a308-4c70c86762ce-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.349830 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.349733 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea5dcba5-3cc5-4452-a308-4c70c86762ce-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.349830 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.349765 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea5dcba5-3cc5-4452-a308-4c70c86762ce-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.349830 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.349792 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea5dcba5-3cc5-4452-a308-4c70c86762ce-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.349928 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.349848 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea5dcba5-3cc5-4452-a308-4c70c86762ce-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.349928 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.349871 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dq5q\" (UniqueName: \"kubernetes.io/projected/ea5dcba5-3cc5-4452-a308-4c70c86762ce-kube-api-access-5dq5q\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.450624 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.450583 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea5dcba5-3cc5-4452-a308-4c70c86762ce-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.450624 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.450624 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea5dcba5-3cc5-4452-a308-4c70c86762ce-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.450935 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.450660 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea5dcba5-3cc5-4452-a308-4c70c86762ce-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.450935 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.450679 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea5dcba5-3cc5-4452-a308-4c70c86762ce-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.450935 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.450699 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea5dcba5-3cc5-4452-a308-4c70c86762ce-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.450935 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.450721 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea5dcba5-3cc5-4452-a308-4c70c86762ce-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.450935 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.450748 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea5dcba5-3cc5-4452-a308-4c70c86762ce-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.450935 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.450807 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea5dcba5-3cc5-4452-a308-4c70c86762ce-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.451238 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.450995 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dq5q\" (UniqueName: \"kubernetes.io/projected/ea5dcba5-3cc5-4452-a308-4c70c86762ce-kube-api-access-5dq5q\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.451238 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.451151 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea5dcba5-3cc5-4452-a308-4c70c86762ce-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.451352 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.451331 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea5dcba5-3cc5-4452-a308-4c70c86762ce-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.451410 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.451349 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea5dcba5-3cc5-4452-a308-4c70c86762ce-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.451496 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.451473 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea5dcba5-3cc5-4452-a308-4c70c86762ce-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.451768 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.451749 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea5dcba5-3cc5-4452-a308-4c70c86762ce-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.453583 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.453489 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea5dcba5-3cc5-4452-a308-4c70c86762ce-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.453583 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.453574 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:39.453796 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.453581 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea5dcba5-3cc5-4452-a308-4c70c86762ce-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.459045 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.459013 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea5dcba5-3cc5-4452-a308-4c70c86762ce-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.459160 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.459078 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dq5q\" (UniqueName: \"kubernetes.io/projected/ea5dcba5-3cc5-4452-a308-4c70c86762ce-kube-api-access-5dq5q\") pod \"maas-default-gateway-openshift-default-5ccb968c66-qxx2n\" (UID: \"ea5dcba5-3cc5-4452-a308-4c70c86762ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.637334 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.637243 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:39.777993 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.777967 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n"] Jul 31 23:05:39.780794 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:05:39.780765 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea5dcba5_3cc5_4452_a308_4c70c86762ce.slice/crio-b7df5ce2d6402f7d6590d7c26c2796865d692aa3f8d16f31a2ce92c748e10e1e WatchSource:0}: Error finding container b7df5ce2d6402f7d6590d7c26c2796865d692aa3f8d16f31a2ce92c748e10e1e: Status 404 returned error can't find the container with id b7df5ce2d6402f7d6590d7c26c2796865d692aa3f8d16f31a2ce92c748e10e1e Jul 31 23:05:39.782884 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.782856 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:05:39.782975 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.782924 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:05:39.782975 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:39.782955 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:05:40.525369 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:40.525331 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" event={"ID":"ea5dcba5-3cc5-4452-a308-4c70c86762ce","Type":"ContainerStarted","Data":"01fce6d90034e41fb9848687ed9b2e042627a43c905ca697f1188b0c084195b4"} Jul 31 23:05:40.525864 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:40.525397 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" event={"ID":"ea5dcba5-3cc5-4452-a308-4c70c86762ce","Type":"ContainerStarted","Data":"b7df5ce2d6402f7d6590d7c26c2796865d692aa3f8d16f31a2ce92c748e10e1e"} Jul 31 23:05:40.547736 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:40.547682 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" podStartSLOduration=1.5476476369999999 podStartE2EDuration="1.547647637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:05:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:05:40.546052178 +0000 UTC m=+497.018496374" watchObservedRunningTime="2026-07-31 23:05:40.547647637 +0000 UTC m=+497.020091819" Jul 31 23:05:40.637779 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:40.637733 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:40.642815 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:40.642788 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:41.530766 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:41.530733 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:41.531677 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:41.531658 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:05:43.707685 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:43.707648 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:43.810711 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:43.810683 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:52.877271 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:52.877216 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:52.992767 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:52.992733 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:53.094308 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:53.094269 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:53.715101 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:53.715067 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:53.799418 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:53.799381 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:05:53.800160 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:53.800141 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-lxr59"] Jul 31 23:05:53.803318 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:53.803303 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-lxr59" Jul 31 23:05:53.805984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:53.805967 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nfv87\"" Jul 31 23:05:53.809695 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:53.809672 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-lxr59"] Jul 31 23:05:53.882672 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:53.882629 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql2rb\" (UniqueName: \"kubernetes.io/projected/4a526f52-8710-4ee1-8e67-ac8ca3908b04-kube-api-access-ql2rb\") pod \"authorino-c548b5d7d-lxr59\" (UID: \"4a526f52-8710-4ee1-8e67-ac8ca3908b04\") " pod="kuadrant-system/authorino-c548b5d7d-lxr59" Jul 31 23:05:53.983491 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:53.983394 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql2rb\" (UniqueName: \"kubernetes.io/projected/4a526f52-8710-4ee1-8e67-ac8ca3908b04-kube-api-access-ql2rb\") pod \"authorino-c548b5d7d-lxr59\" (UID: \"4a526f52-8710-4ee1-8e67-ac8ca3908b04\") " pod="kuadrant-system/authorino-c548b5d7d-lxr59" Jul 31 23:05:53.992163 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:53.992133 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql2rb\" (UniqueName: \"kubernetes.io/projected/4a526f52-8710-4ee1-8e67-ac8ca3908b04-kube-api-access-ql2rb\") pod \"authorino-c548b5d7d-lxr59\" (UID: \"4a526f52-8710-4ee1-8e67-ac8ca3908b04\") " pod="kuadrant-system/authorino-c548b5d7d-lxr59" Jul 31 23:05:54.114144 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:54.114111 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-lxr59" Jul 31 23:05:54.249974 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:54.249886 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-lxr59"] Jul 31 23:05:54.253249 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:05:54.253211 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a526f52_8710_4ee1_8e67_ac8ca3908b04.slice/crio-e1cc7f944f296688c52d0e831075f2cc39a0a72d4eb87ea478a5bead3348410a WatchSource:0}: Error finding container e1cc7f944f296688c52d0e831075f2cc39a0a72d4eb87ea478a5bead3348410a: Status 404 returned error can't find the container with id e1cc7f944f296688c52d0e831075f2cc39a0a72d4eb87ea478a5bead3348410a Jul 31 23:05:54.594956 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:54.594919 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-lxr59" event={"ID":"4a526f52-8710-4ee1-8e67-ac8ca3908b04","Type":"ContainerStarted","Data":"e1cc7f944f296688c52d0e831075f2cc39a0a72d4eb87ea478a5bead3348410a"} Jul 31 23:05:57.613369 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:57.613318 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-lxr59" event={"ID":"4a526f52-8710-4ee1-8e67-ac8ca3908b04","Type":"ContainerStarted","Data":"75e1724bb9f51324f9f537c3036d629ca09c5b303162538b1782d69f4adc6b89"} Jul 31 23:05:57.628511 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:05:57.628455 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-lxr59" podStartSLOduration=2.073038968 podStartE2EDuration="4.628435776s" totalImagesPullingTime="2.555396808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:05:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:05:57.62731092 +0000 UTC m=+514.099755103" watchObservedRunningTime="2026-07-31 23:05:57.628435776 +0000 UTC m=+514.100879958" Jul 31 23:06:01.403562 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:01.403524 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:06:12.918840 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:12.918798 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:06:13.021772 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:13.021738 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:06:13.393072 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:13.393035 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:06:25.629370 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:25.629277 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:06:25.845935 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:25.845901 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-hsnj4"] Jul 31 23:06:25.849081 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:25.849058 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-hsnj4" Jul 31 23:06:25.857249 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:25.857220 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-hsnj4"] Jul 31 23:06:25.892109 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:25.892018 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rljqm\" (UniqueName: \"kubernetes.io/projected/2736f94c-6e31-42a4-a245-6bf7cb553683-kube-api-access-rljqm\") pod \"authorino-5dc7ccb5f-hsnj4\" (UID: \"2736f94c-6e31-42a4-a245-6bf7cb553683\") " pod="kuadrant-system/authorino-5dc7ccb5f-hsnj4" Jul 31 23:06:25.993276 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:25.993239 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rljqm\" (UniqueName: \"kubernetes.io/projected/2736f94c-6e31-42a4-a245-6bf7cb553683-kube-api-access-rljqm\") pod \"authorino-5dc7ccb5f-hsnj4\" (UID: \"2736f94c-6e31-42a4-a245-6bf7cb553683\") " pod="kuadrant-system/authorino-5dc7ccb5f-hsnj4" Jul 31 23:06:26.005370 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.005330 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rljqm\" (UniqueName: \"kubernetes.io/projected/2736f94c-6e31-42a4-a245-6bf7cb553683-kube-api-access-rljqm\") pod \"authorino-5dc7ccb5f-hsnj4\" (UID: \"2736f94c-6e31-42a4-a245-6bf7cb553683\") " pod="kuadrant-system/authorino-5dc7ccb5f-hsnj4" Jul 31 23:06:26.059535 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.059475 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:06:26.090745 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.090708 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-hsnj4"] Jul 31 23:06:26.091014 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.091000 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-hsnj4" Jul 31 23:06:26.132776 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.132745 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-64f76c56f8-4rql6"] Jul 31 23:06:26.136968 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.136925 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64f76c56f8-4rql6" Jul 31 23:06:26.138991 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.138964 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64f76c56f8-4rql6"] Jul 31 23:06:26.179487 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.179449 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64f76c56f8-4rql6"] Jul 31 23:06:26.179896 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:06:26.179860 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-8vc49], unattached volumes=[], failed to process volumes=[kube-api-access-8vc49]: context canceled" pod="kuadrant-system/authorino-64f76c56f8-4rql6" podUID="3d3c14d7-6ce1-48b3-ae91-504db308712b" Jul 31 23:06:26.195190 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.195148 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vc49\" (UniqueName: \"kubernetes.io/projected/3d3c14d7-6ce1-48b3-ae91-504db308712b-kube-api-access-8vc49\") pod \"authorino-64f76c56f8-4rql6\" (UID: \"3d3c14d7-6ce1-48b3-ae91-504db308712b\") " pod="kuadrant-system/authorino-64f76c56f8-4rql6" Jul 31 23:06:26.216200 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.216164 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-78b85c8676-k5qd9"] Jul 31 23:06:26.220374 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.220344 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78b85c8676-k5qd9" Jul 31 23:06:26.223045 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.223019 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 31 23:06:26.226756 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.226725 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78b85c8676-k5qd9"] Jul 31 23:06:26.249260 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.249072 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-hsnj4"] Jul 31 23:06:26.251598 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:06:26.251564 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2736f94c_6e31_42a4_a245_6bf7cb553683.slice/crio-b0f26644af840db852a706dcf324a11fbb0f56892e97edd35eef884068310712 WatchSource:0}: Error finding container b0f26644af840db852a706dcf324a11fbb0f56892e97edd35eef884068310712: Status 404 returned error can't find the container with id b0f26644af840db852a706dcf324a11fbb0f56892e97edd35eef884068310712 Jul 31 23:06:26.296244 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.296203 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vc49\" (UniqueName: \"kubernetes.io/projected/3d3c14d7-6ce1-48b3-ae91-504db308712b-kube-api-access-8vc49\") pod \"authorino-64f76c56f8-4rql6\" (UID: \"3d3c14d7-6ce1-48b3-ae91-504db308712b\") " pod="kuadrant-system/authorino-64f76c56f8-4rql6" Jul 31 23:06:26.296439 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.296269 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/acf8983c-ad7c-482d-8877-aebb71b6595e-tls-cert\") pod \"authorino-78b85c8676-k5qd9\" (UID: \"acf8983c-ad7c-482d-8877-aebb71b6595e\") " pod="kuadrant-system/authorino-78b85c8676-k5qd9" Jul 31 23:06:26.296439 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.296310 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g4km\" (UniqueName: \"kubernetes.io/projected/acf8983c-ad7c-482d-8877-aebb71b6595e-kube-api-access-5g4km\") pod \"authorino-78b85c8676-k5qd9\" (UID: \"acf8983c-ad7c-482d-8877-aebb71b6595e\") " pod="kuadrant-system/authorino-78b85c8676-k5qd9" Jul 31 23:06:26.304821 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.304782 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vc49\" (UniqueName: \"kubernetes.io/projected/3d3c14d7-6ce1-48b3-ae91-504db308712b-kube-api-access-8vc49\") pod \"authorino-64f76c56f8-4rql6\" (UID: \"3d3c14d7-6ce1-48b3-ae91-504db308712b\") " pod="kuadrant-system/authorino-64f76c56f8-4rql6" Jul 31 23:06:26.397287 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.397249 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g4km\" (UniqueName: \"kubernetes.io/projected/acf8983c-ad7c-482d-8877-aebb71b6595e-kube-api-access-5g4km\") pod \"authorino-78b85c8676-k5qd9\" (UID: \"acf8983c-ad7c-482d-8877-aebb71b6595e\") " pod="kuadrant-system/authorino-78b85c8676-k5qd9" Jul 31 23:06:26.397466 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.397372 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/acf8983c-ad7c-482d-8877-aebb71b6595e-tls-cert\") pod \"authorino-78b85c8676-k5qd9\" (UID: \"acf8983c-ad7c-482d-8877-aebb71b6595e\") " pod="kuadrant-system/authorino-78b85c8676-k5qd9" Jul 31 23:06:26.399864 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.399812 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/acf8983c-ad7c-482d-8877-aebb71b6595e-tls-cert\") pod \"authorino-78b85c8676-k5qd9\" (UID: \"acf8983c-ad7c-482d-8877-aebb71b6595e\") " pod="kuadrant-system/authorino-78b85c8676-k5qd9" Jul 31 23:06:26.405743 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.405714 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g4km\" (UniqueName: \"kubernetes.io/projected/acf8983c-ad7c-482d-8877-aebb71b6595e-kube-api-access-5g4km\") pod \"authorino-78b85c8676-k5qd9\" (UID: \"acf8983c-ad7c-482d-8877-aebb71b6595e\") " pod="kuadrant-system/authorino-78b85c8676-k5qd9" Jul 31 23:06:26.532910 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.532877 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78b85c8676-k5qd9" Jul 31 23:06:26.677438 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.677414 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78b85c8676-k5qd9"] Jul 31 23:06:26.679524 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:06:26.679479 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podacf8983c_ad7c_482d_8877_aebb71b6595e.slice/crio-8272f7e935dfc64e9d5c4c4b37e8bfbf811dd4051f9ba760ca068ea6bf5f80fc WatchSource:0}: Error finding container 8272f7e935dfc64e9d5c4c4b37e8bfbf811dd4051f9ba760ca068ea6bf5f80fc: Status 404 returned error can't find the container with id 8272f7e935dfc64e9d5c4c4b37e8bfbf811dd4051f9ba760ca068ea6bf5f80fc Jul 31 23:06:26.755496 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.755453 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-hsnj4" event={"ID":"2736f94c-6e31-42a4-a245-6bf7cb553683","Type":"ContainerStarted","Data":"b0f26644af840db852a706dcf324a11fbb0f56892e97edd35eef884068310712"} Jul 31 23:06:26.756788 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.756758 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78b85c8676-k5qd9" event={"ID":"acf8983c-ad7c-482d-8877-aebb71b6595e","Type":"ContainerStarted","Data":"8272f7e935dfc64e9d5c4c4b37e8bfbf811dd4051f9ba760ca068ea6bf5f80fc"} Jul 31 23:06:26.756788 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.756769 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64f76c56f8-4rql6" Jul 31 23:06:26.766627 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.766600 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64f76c56f8-4rql6" Jul 31 23:06:26.802479 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.802448 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vc49\" (UniqueName: \"kubernetes.io/projected/3d3c14d7-6ce1-48b3-ae91-504db308712b-kube-api-access-8vc49\") pod \"3d3c14d7-6ce1-48b3-ae91-504db308712b\" (UID: \"3d3c14d7-6ce1-48b3-ae91-504db308712b\") " Jul 31 23:06:26.804555 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.804523 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d3c14d7-6ce1-48b3-ae91-504db308712b-kube-api-access-8vc49" (OuterVolumeSpecName: "kube-api-access-8vc49") pod "3d3c14d7-6ce1-48b3-ae91-504db308712b" (UID: "3d3c14d7-6ce1-48b3-ae91-504db308712b"). InnerVolumeSpecName "kube-api-access-8vc49". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:06:26.903718 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:26.903680 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vc49\" (UniqueName: \"kubernetes.io/projected/3d3c14d7-6ce1-48b3-ae91-504db308712b-kube-api-access-8vc49\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:06:27.762870 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:27.762835 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78b85c8676-k5qd9" event={"ID":"acf8983c-ad7c-482d-8877-aebb71b6595e","Type":"ContainerStarted","Data":"051f2c4d1152a024272e988623d1410ebf54c0b0492e3a04505285b25106e41a"} Jul 31 23:06:27.764300 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:27.764282 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64f76c56f8-4rql6" Jul 31 23:06:27.764374 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:27.764336 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5dc7ccb5f-hsnj4" podUID="2736f94c-6e31-42a4-a245-6bf7cb553683" containerName="authorino" containerID="cri-o://2a08ac25df01d5bee8c3cb539b340c27ee92e3b6774c2b25e273d34e2d4c831f" gracePeriod=30 Jul 31 23:06:27.764440 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:27.764282 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-hsnj4" event={"ID":"2736f94c-6e31-42a4-a245-6bf7cb553683","Type":"ContainerStarted","Data":"2a08ac25df01d5bee8c3cb539b340c27ee92e3b6774c2b25e273d34e2d4c831f"} Jul 31 23:06:27.786563 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:27.780959 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-78b85c8676-k5qd9" podStartSLOduration=1.32629565 podStartE2EDuration="1.780938052s" totalImagesPullingTime="454.642402ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:06:27.779333344 +0000 UTC m=+544.251777528" watchObservedRunningTime="2026-07-31 23:06:27.780938052 +0000 UTC m=+544.253382236" Jul 31 23:06:27.808933 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:27.808895 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-lxr59"] Jul 31 23:06:27.809140 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:27.809115 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-lxr59" podUID="4a526f52-8710-4ee1-8e67-ac8ca3908b04" containerName="authorino" containerID="cri-o://75e1724bb9f51324f9f537c3036d629ca09c5b303162538b1782d69f4adc6b89" gracePeriod=30 Jul 31 23:06:27.818435 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:27.818327 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64f76c56f8-4rql6"] Jul 31 23:06:27.829536 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:27.829484 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-64f76c56f8-4rql6"] Jul 31 23:06:27.848361 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:27.848298 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5dc7ccb5f-hsnj4" podStartSLOduration=2.060910696 podStartE2EDuration="2.848275111s" totalImagesPullingTime="787.364415ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:06:27.846662602 +0000 UTC m=+544.319106807" watchObservedRunningTime="2026-07-31 23:06:27.848275111 +0000 UTC m=+544.320719294" Jul 31 23:06:28.052811 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.052778 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-hsnj4" Jul 31 23:06:28.088160 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.088133 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-lxr59" Jul 31 23:06:28.115875 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.115843 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql2rb\" (UniqueName: \"kubernetes.io/projected/4a526f52-8710-4ee1-8e67-ac8ca3908b04-kube-api-access-ql2rb\") pod \"4a526f52-8710-4ee1-8e67-ac8ca3908b04\" (UID: \"4a526f52-8710-4ee1-8e67-ac8ca3908b04\") " Jul 31 23:06:28.115875 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.115874 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rljqm\" (UniqueName: \"kubernetes.io/projected/2736f94c-6e31-42a4-a245-6bf7cb553683-kube-api-access-rljqm\") pod \"2736f94c-6e31-42a4-a245-6bf7cb553683\" (UID: \"2736f94c-6e31-42a4-a245-6bf7cb553683\") " Jul 31 23:06:28.118066 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.118037 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2736f94c-6e31-42a4-a245-6bf7cb553683-kube-api-access-rljqm" (OuterVolumeSpecName: "kube-api-access-rljqm") pod "2736f94c-6e31-42a4-a245-6bf7cb553683" (UID: "2736f94c-6e31-42a4-a245-6bf7cb553683"). InnerVolumeSpecName "kube-api-access-rljqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:06:28.118164 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.118103 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a526f52-8710-4ee1-8e67-ac8ca3908b04-kube-api-access-ql2rb" (OuterVolumeSpecName: "kube-api-access-ql2rb") pod "4a526f52-8710-4ee1-8e67-ac8ca3908b04" (UID: "4a526f52-8710-4ee1-8e67-ac8ca3908b04"). InnerVolumeSpecName "kube-api-access-ql2rb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:06:28.132607 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.132569 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d3c14d7-6ce1-48b3-ae91-504db308712b" path="/var/lib/kubelet/pods/3d3c14d7-6ce1-48b3-ae91-504db308712b/volumes" Jul 31 23:06:28.217191 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.217165 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql2rb\" (UniqueName: \"kubernetes.io/projected/4a526f52-8710-4ee1-8e67-ac8ca3908b04-kube-api-access-ql2rb\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:06:28.217191 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.217188 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rljqm\" (UniqueName: \"kubernetes.io/projected/2736f94c-6e31-42a4-a245-6bf7cb553683-kube-api-access-rljqm\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:06:28.771226 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.771191 2561 generic.go:356] "Generic (PLEG): container finished" podID="4a526f52-8710-4ee1-8e67-ac8ca3908b04" containerID="75e1724bb9f51324f9f537c3036d629ca09c5b303162538b1782d69f4adc6b89" exitCode=0 Jul 31 23:06:28.771775 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.771257 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-lxr59" Jul 31 23:06:28.771775 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.771288 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-lxr59" event={"ID":"4a526f52-8710-4ee1-8e67-ac8ca3908b04","Type":"ContainerDied","Data":"75e1724bb9f51324f9f537c3036d629ca09c5b303162538b1782d69f4adc6b89"} Jul 31 23:06:28.771775 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.771327 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-lxr59" event={"ID":"4a526f52-8710-4ee1-8e67-ac8ca3908b04","Type":"ContainerDied","Data":"e1cc7f944f296688c52d0e831075f2cc39a0a72d4eb87ea478a5bead3348410a"} Jul 31 23:06:28.771775 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.771345 2561 scope.go:117] "RemoveContainer" containerID="75e1724bb9f51324f9f537c3036d629ca09c5b303162538b1782d69f4adc6b89" Jul 31 23:06:28.772759 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.772738 2561 generic.go:356] "Generic (PLEG): container finished" podID="2736f94c-6e31-42a4-a245-6bf7cb553683" containerID="2a08ac25df01d5bee8c3cb539b340c27ee92e3b6774c2b25e273d34e2d4c831f" exitCode=0 Jul 31 23:06:28.772863 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.772786 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-hsnj4" Jul 31 23:06:28.772863 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.772819 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-hsnj4" event={"ID":"2736f94c-6e31-42a4-a245-6bf7cb553683","Type":"ContainerDied","Data":"2a08ac25df01d5bee8c3cb539b340c27ee92e3b6774c2b25e273d34e2d4c831f"} Jul 31 23:06:28.772863 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.772855 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-hsnj4" event={"ID":"2736f94c-6e31-42a4-a245-6bf7cb553683","Type":"ContainerDied","Data":"b0f26644af840db852a706dcf324a11fbb0f56892e97edd35eef884068310712"} Jul 31 23:06:28.780263 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.780239 2561 scope.go:117] "RemoveContainer" containerID="75e1724bb9f51324f9f537c3036d629ca09c5b303162538b1782d69f4adc6b89" Jul 31 23:06:28.780574 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:06:28.780556 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75e1724bb9f51324f9f537c3036d629ca09c5b303162538b1782d69f4adc6b89\": container with ID starting with 75e1724bb9f51324f9f537c3036d629ca09c5b303162538b1782d69f4adc6b89 not found: ID does not exist" containerID="75e1724bb9f51324f9f537c3036d629ca09c5b303162538b1782d69f4adc6b89" Jul 31 23:06:28.780640 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.780582 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75e1724bb9f51324f9f537c3036d629ca09c5b303162538b1782d69f4adc6b89"} err="failed to get container status \"75e1724bb9f51324f9f537c3036d629ca09c5b303162538b1782d69f4adc6b89\": rpc error: code = NotFound desc = could not find container \"75e1724bb9f51324f9f537c3036d629ca09c5b303162538b1782d69f4adc6b89\": container with ID starting with 75e1724bb9f51324f9f537c3036d629ca09c5b303162538b1782d69f4adc6b89 not found: ID does not exist" Jul 31 23:06:28.780640 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.780604 2561 scope.go:117] "RemoveContainer" containerID="2a08ac25df01d5bee8c3cb539b340c27ee92e3b6774c2b25e273d34e2d4c831f" Jul 31 23:06:28.787487 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.787461 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-lxr59"] Jul 31 23:06:28.788846 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.788822 2561 scope.go:117] "RemoveContainer" containerID="2a08ac25df01d5bee8c3cb539b340c27ee92e3b6774c2b25e273d34e2d4c831f" Jul 31 23:06:28.789392 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:06:28.789367 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a08ac25df01d5bee8c3cb539b340c27ee92e3b6774c2b25e273d34e2d4c831f\": container with ID starting with 2a08ac25df01d5bee8c3cb539b340c27ee92e3b6774c2b25e273d34e2d4c831f not found: ID does not exist" containerID="2a08ac25df01d5bee8c3cb539b340c27ee92e3b6774c2b25e273d34e2d4c831f" Jul 31 23:06:28.789485 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.789401 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a08ac25df01d5bee8c3cb539b340c27ee92e3b6774c2b25e273d34e2d4c831f"} err="failed to get container status \"2a08ac25df01d5bee8c3cb539b340c27ee92e3b6774c2b25e273d34e2d4c831f\": rpc error: code = NotFound desc = could not find container \"2a08ac25df01d5bee8c3cb539b340c27ee92e3b6774c2b25e273d34e2d4c831f\": container with ID starting with 2a08ac25df01d5bee8c3cb539b340c27ee92e3b6774c2b25e273d34e2d4c831f not found: ID does not exist" Jul 31 23:06:28.791238 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.791220 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-lxr59"] Jul 31 23:06:28.805071 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.805032 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-hsnj4"] Jul 31 23:06:28.808590 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:28.808565 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-hsnj4"] Jul 31 23:06:30.132929 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:30.132898 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2736f94c-6e31-42a4-a245-6bf7cb553683" path="/var/lib/kubelet/pods/2736f94c-6e31-42a4-a245-6bf7cb553683/volumes" Jul 31 23:06:30.133315 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:30.133250 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a526f52-8710-4ee1-8e67-ac8ca3908b04" path="/var/lib/kubelet/pods/4a526f52-8710-4ee1-8e67-ac8ca3908b04/volumes" Jul 31 23:06:32.263381 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:32.263345 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:06:32.362792 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:32.362760 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:06:32.447252 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:32.447201 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:06:32.509062 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:32.509022 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:06:34.370769 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:34.370713 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:06:34.481693 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:06:34.481655 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:07:03.635228 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:03.635195 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:07:03.734270 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:03.734238 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:07:05.127086 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.127051 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:07:05.372252 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.372220 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:07:05.813764 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.813732 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:07:05.885084 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.885051 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:07:05.916547 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.916495 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7"] Jul 31 23:07:05.917020 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.917000 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a526f52-8710-4ee1-8e67-ac8ca3908b04" containerName="authorino" Jul 31 23:07:05.917115 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.917022 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a526f52-8710-4ee1-8e67-ac8ca3908b04" containerName="authorino" Jul 31 23:07:05.917115 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.917038 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2736f94c-6e31-42a4-a245-6bf7cb553683" containerName="authorino" Jul 31 23:07:05.917115 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.917046 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2736f94c-6e31-42a4-a245-6bf7cb553683" containerName="authorino" Jul 31 23:07:05.917266 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.917160 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a526f52-8710-4ee1-8e67-ac8ca3908b04" containerName="authorino" Jul 31 23:07:05.917266 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.917180 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2736f94c-6e31-42a4-a245-6bf7cb553683" containerName="authorino" Jul 31 23:07:05.920904 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.920879 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7" Jul 31 23:07:05.923414 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.923384 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 31 23:07:05.923561 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.923546 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-mhmmr\"" Jul 31 23:07:05.930541 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.930516 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7"] Jul 31 23:07:05.981169 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.981130 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qccnq\" (UniqueName: \"kubernetes.io/projected/472aafd0-2786-4a94-9af0-d433ebed2139-kube-api-access-qccnq\") pod \"payload-pre-processing-687dbb6cf6-4jmd7\" (UID: \"472aafd0-2786-4a94-9af0-d433ebed2139\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7" Jul 31 23:07:05.981169 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:05.981168 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/472aafd0-2786-4a94-9af0-d433ebed2139-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4jmd7\" (UID: \"472aafd0-2786-4a94-9af0-d433ebed2139\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7" Jul 31 23:07:06.082702 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:06.082603 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qccnq\" (UniqueName: \"kubernetes.io/projected/472aafd0-2786-4a94-9af0-d433ebed2139-kube-api-access-qccnq\") pod \"payload-pre-processing-687dbb6cf6-4jmd7\" (UID: \"472aafd0-2786-4a94-9af0-d433ebed2139\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7" Jul 31 23:07:06.082702 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:06.082671 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/472aafd0-2786-4a94-9af0-d433ebed2139-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4jmd7\" (UID: \"472aafd0-2786-4a94-9af0-d433ebed2139\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7" Jul 31 23:07:06.083822 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:06.083802 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/472aafd0-2786-4a94-9af0-d433ebed2139-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4jmd7\" (UID: \"472aafd0-2786-4a94-9af0-d433ebed2139\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7" Jul 31 23:07:06.095229 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:06.095198 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qccnq\" (UniqueName: \"kubernetes.io/projected/472aafd0-2786-4a94-9af0-d433ebed2139-kube-api-access-qccnq\") pod \"payload-pre-processing-687dbb6cf6-4jmd7\" (UID: \"472aafd0-2786-4a94-9af0-d433ebed2139\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7" Jul 31 23:07:06.233129 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:06.233093 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7" Jul 31 23:07:06.377725 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:06.377688 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7"] Jul 31 23:07:06.379589 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:07:06.379543 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod472aafd0_2786_4a94_9af0_d433ebed2139.slice/crio-b8e8ecdef17396161c7ab4758fb0e349ba343709c5234b019bef2e6ca5ef8b03 WatchSource:0}: Error finding container b8e8ecdef17396161c7ab4758fb0e349ba343709c5234b019bef2e6ca5ef8b03: Status 404 returned error can't find the container with id b8e8ecdef17396161c7ab4758fb0e349ba343709c5234b019bef2e6ca5ef8b03 Jul 31 23:07:06.976863 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:06.976825 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7" event={"ID":"472aafd0-2786-4a94-9af0-d433ebed2139","Type":"ContainerStarted","Data":"b8e8ecdef17396161c7ab4758fb0e349ba343709c5234b019bef2e6ca5ef8b03"} Jul 31 23:07:08.990567 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:08.990534 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7" event={"ID":"472aafd0-2786-4a94-9af0-d433ebed2139","Type":"ContainerStarted","Data":"d374dc5c9505fdec9e237fe9103ddf4a14d0809810ead60f41aced82a32585c3"} Jul 31 23:07:08.990956 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:08.990622 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7" Jul 31 23:07:09.006898 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:09.006847 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7" podStartSLOduration=2.084405897 podStartE2EDuration="4.006828791s" totalImagesPullingTime="1.922422894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:07:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:07:09.005454892 +0000 UTC m=+585.477899075" watchObservedRunningTime="2026-07-31 23:07:09.006828791 +0000 UTC m=+585.479272975" Jul 31 23:07:14.492814 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:14.492776 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:07:14.599931 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:14.599888 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:07:15.022973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:15.022938 2561 generic.go:356] "Generic (PLEG): container finished" podID="ea5dcba5-3cc5-4452-a308-4c70c86762ce" containerID="01fce6d90034e41fb9848687ed9b2e042627a43c905ca697f1188b0c084195b4" exitCode=0 Jul 31 23:07:15.023177 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:15.023005 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" event={"ID":"ea5dcba5-3cc5-4452-a308-4c70c86762ce","Type":"ContainerDied","Data":"01fce6d90034e41fb9848687ed9b2e042627a43c905ca697f1188b0c084195b4"} Jul 31 23:07:15.023432 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:15.023418 2561 scope.go:117] "RemoveContainer" containerID="01fce6d90034e41fb9848687ed9b2e042627a43c905ca697f1188b0c084195b4" Jul 31 23:07:15.024463 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:15.024437 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:07:15.024545 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:15.024531 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:07:15.024590 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:15.024572 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:07:16.030381 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:16.030348 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" event={"ID":"ea5dcba5-3cc5-4452-a308-4c70c86762ce","Type":"ContainerStarted","Data":"b8284f4105c2788a886a2a70bcd662b35fd9c60fa329cb85f6865af95ff8efea"} Jul 31 23:07:16.637913 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:16.637862 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:07:16.638146 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:16.638040 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-qxx2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 31 23:07:16.638146 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:16.638090 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" podUID="ea5dcba5-3cc5-4452-a308-4c70c86762ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 31 23:07:17.638310 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:17.638271 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-qxx2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 31 23:07:17.638849 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:17.638343 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" podUID="ea5dcba5-3cc5-4452-a308-4c70c86762ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 31 23:07:18.638036 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:18.637998 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-qxx2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 31 23:07:18.638357 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:18.638327 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" podUID="ea5dcba5-3cc5-4452-a308-4c70c86762ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 31 23:07:18.921134 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:18.921056 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:07:19.638131 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:19.638088 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-qxx2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 31 23:07:19.638448 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:19.638419 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" podUID="ea5dcba5-3cc5-4452-a308-4c70c86762ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 31 23:07:19.998091 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:19.998015 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4jmd7" Jul 31 23:07:20.640001 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:20.639956 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-qxx2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 31 23:07:20.640536 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:20.640084 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" podUID="ea5dcba5-3cc5-4452-a308-4c70c86762ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 31 23:07:21.590343 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.589999 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r"] Jul 31 23:07:21.594550 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.594496 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.599995 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.599955 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 31 23:07:21.600173 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.599958 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xm9gz\"" Jul 31 23:07:21.600378 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.600353 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 31 23:07:21.601096 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.601076 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 31 23:07:21.607707 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.607542 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r"] Jul 31 23:07:21.619191 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.619161 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:07:21.651934 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.651898 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:07:21.652396 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.652263 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:07:21.653138 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.653112 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-qxx2n" Jul 31 23:07:21.742458 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.742412 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9640ef64-fe7a-4f6e-865f-11b9e494a47f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.742689 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.742485 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp857\" (UniqueName: \"kubernetes.io/projected/9640ef64-fe7a-4f6e-865f-11b9e494a47f-kube-api-access-xp857\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.742689 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.742575 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9640ef64-fe7a-4f6e-865f-11b9e494a47f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.742689 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.742604 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9640ef64-fe7a-4f6e-865f-11b9e494a47f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.742838 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.742771 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9640ef64-fe7a-4f6e-865f-11b9e494a47f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.742838 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.742821 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9640ef64-fe7a-4f6e-865f-11b9e494a47f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.843811 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.843704 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9640ef64-fe7a-4f6e-865f-11b9e494a47f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.843811 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.843743 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9640ef64-fe7a-4f6e-865f-11b9e494a47f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.843811 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.843809 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9640ef64-fe7a-4f6e-865f-11b9e494a47f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.844100 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.843839 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9640ef64-fe7a-4f6e-865f-11b9e494a47f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.844100 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.843864 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9640ef64-fe7a-4f6e-865f-11b9e494a47f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.844100 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.843993 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp857\" (UniqueName: \"kubernetes.io/projected/9640ef64-fe7a-4f6e-865f-11b9e494a47f-kube-api-access-xp857\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.844260 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.844207 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9640ef64-fe7a-4f6e-865f-11b9e494a47f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.844260 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.844227 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9640ef64-fe7a-4f6e-865f-11b9e494a47f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.844375 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.844262 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9640ef64-fe7a-4f6e-865f-11b9e494a47f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.846150 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.846121 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9640ef64-fe7a-4f6e-865f-11b9e494a47f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.846562 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.846542 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9640ef64-fe7a-4f6e-865f-11b9e494a47f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.852321 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.852290 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp857\" (UniqueName: \"kubernetes.io/projected/9640ef64-fe7a-4f6e-865f-11b9e494a47f-kube-api-access-xp857\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ds29r\" (UID: \"9640ef64-fe7a-4f6e-865f-11b9e494a47f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:21.911394 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:21.911358 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:22.059645 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:22.059455 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r"] Jul 31 23:07:22.064987 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:07:22.063171 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9640ef64_fe7a_4f6e_865f_11b9e494a47f.slice/crio-b0e7d45c0ca08eb5fb761ef217e0e2a00f09cb213d3756f69440a6b2c12486b8 WatchSource:0}: Error finding container b0e7d45c0ca08eb5fb761ef217e0e2a00f09cb213d3756f69440a6b2c12486b8: Status 404 returned error can't find the container with id b0e7d45c0ca08eb5fb761ef217e0e2a00f09cb213d3756f69440a6b2c12486b8 Jul 31 23:07:22.075720 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:22.075688 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" event={"ID":"9640ef64-fe7a-4f6e-865f-11b9e494a47f","Type":"ContainerStarted","Data":"b0e7d45c0ca08eb5fb761ef217e0e2a00f09cb213d3756f69440a6b2c12486b8"} Jul 31 23:07:24.089675 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:24.089630 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" event={"ID":"9640ef64-fe7a-4f6e-865f-11b9e494a47f","Type":"ContainerStarted","Data":"37a97cd5bd8561683bf6f08ab3ef2472244486f1d587636697445d014e973c48"} Jul 31 23:07:24.109520 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:24.109448 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" podStartSLOduration=1.9501762120000001 podStartE2EDuration="3.10943028s" totalImagesPullingTime="1.159254068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:07:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:07:24.108101565 +0000 UTC m=+600.580545750" watchObservedRunningTime="2026-07-31 23:07:24.10943028 +0000 UTC m=+600.581874462" Jul 31 23:07:28.577848 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:28.577811 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:07:31.912397 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:31.912361 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:31.912825 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:31.912416 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:31.925237 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:31.925202 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:32.142801 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:32.142771 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ds29r" Jul 31 23:07:34.886291 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:34.886256 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9"] Jul 31 23:07:34.891399 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:34.891379 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:34.894131 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:34.894109 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 31 23:07:34.900331 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:34.900307 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9"] Jul 31 23:07:34.912987 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:34.912959 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:07:35.069778 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.069737 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m54l\" (UniqueName: \"kubernetes.io/projected/c62841fc-d0b7-40e3-a07c-b79e36d0129c-kube-api-access-5m54l\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.069984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.069825 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c62841fc-d0b7-40e3-a07c-b79e36d0129c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.069984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.069879 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c62841fc-d0b7-40e3-a07c-b79e36d0129c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.069984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.069956 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c62841fc-d0b7-40e3-a07c-b79e36d0129c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.070119 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.069992 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c62841fc-d0b7-40e3-a07c-b79e36d0129c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.070119 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.070078 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c62841fc-d0b7-40e3-a07c-b79e36d0129c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.171477 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.171384 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c62841fc-d0b7-40e3-a07c-b79e36d0129c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.171477 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.171427 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c62841fc-d0b7-40e3-a07c-b79e36d0129c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.171477 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.171468 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c62841fc-d0b7-40e3-a07c-b79e36d0129c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.171763 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.171491 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m54l\" (UniqueName: \"kubernetes.io/projected/c62841fc-d0b7-40e3-a07c-b79e36d0129c-kube-api-access-5m54l\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.171763 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.171567 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c62841fc-d0b7-40e3-a07c-b79e36d0129c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.171763 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.171609 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c62841fc-d0b7-40e3-a07c-b79e36d0129c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.171892 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.171865 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c62841fc-d0b7-40e3-a07c-b79e36d0129c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.171931 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.171902 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c62841fc-d0b7-40e3-a07c-b79e36d0129c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.171990 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.171971 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c62841fc-d0b7-40e3-a07c-b79e36d0129c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.173765 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.173743 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c62841fc-d0b7-40e3-a07c-b79e36d0129c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.174124 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.174106 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c62841fc-d0b7-40e3-a07c-b79e36d0129c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.179370 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.179350 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m54l\" (UniqueName: \"kubernetes.io/projected/c62841fc-d0b7-40e3-a07c-b79e36d0129c-kube-api-access-5m54l\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9\" (UID: \"c62841fc-d0b7-40e3-a07c-b79e36d0129c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.203465 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.203426 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:35.343843 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.343811 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9"] Jul 31 23:07:35.345434 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:07:35.345409 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc62841fc_d0b7_40e3_a07c_b79e36d0129c.slice/crio-ee1130fb7c49d9a07ce1c2b550f238895c98cf9a60bad0851db1f4cf858abd74 WatchSource:0}: Error finding container ee1130fb7c49d9a07ce1c2b550f238895c98cf9a60bad0851db1f4cf858abd74: Status 404 returned error can't find the container with id ee1130fb7c49d9a07ce1c2b550f238895c98cf9a60bad0851db1f4cf858abd74 Jul 31 23:07:35.347485 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:35.347462 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 31 23:07:36.149456 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:36.149414 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" event={"ID":"c62841fc-d0b7-40e3-a07c-b79e36d0129c","Type":"ContainerStarted","Data":"2861d4089bf9bb8eaf7e21dffae0e7f0f3fd8cdb77b470afa27590edfb9ba6a4"} Jul 31 23:07:36.149874 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:36.149474 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" event={"ID":"c62841fc-d0b7-40e3-a07c-b79e36d0129c","Type":"ContainerStarted","Data":"ee1130fb7c49d9a07ce1c2b550f238895c98cf9a60bad0851db1f4cf858abd74"} Jul 31 23:07:36.167122 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:36.167072 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" podStartSLOduration=1.981151191 podStartE2EDuration="2.167056366s" totalImagesPullingTime="185.905175ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:07:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:07:36.165448148 +0000 UTC m=+612.637892335" watchObservedRunningTime="2026-07-31 23:07:36.167056366 +0000 UTC m=+612.639500549" Jul 31 23:07:45.203614 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:45.203575 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:45.204002 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:45.203638 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:45.216152 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:45.216129 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:46.213925 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:46.213891 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9" Jul 31 23:07:57.699143 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.699099 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7"] Jul 31 23:07:57.702931 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.702913 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.705609 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.705593 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 31 23:07:57.712878 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.712855 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7"] Jul 31 23:07:57.784990 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.784955 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/580d3d00-c965-4b73-8141-3203a0637ae4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.785185 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.784997 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6zgg\" (UniqueName: \"kubernetes.io/projected/580d3d00-c965-4b73-8141-3203a0637ae4-kube-api-access-x6zgg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.785185 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.785026 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/580d3d00-c965-4b73-8141-3203a0637ae4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.785185 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.785170 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/580d3d00-c965-4b73-8141-3203a0637ae4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.785336 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.785207 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/580d3d00-c965-4b73-8141-3203a0637ae4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.785336 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.785303 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/580d3d00-c965-4b73-8141-3203a0637ae4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.796526 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.796471 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:07:57.886203 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.886166 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/580d3d00-c965-4b73-8141-3203a0637ae4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.886391 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.886223 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/580d3d00-c965-4b73-8141-3203a0637ae4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.886391 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.886317 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/580d3d00-c965-4b73-8141-3203a0637ae4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.886391 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.886342 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6zgg\" (UniqueName: \"kubernetes.io/projected/580d3d00-c965-4b73-8141-3203a0637ae4-kube-api-access-x6zgg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.886391 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.886369 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/580d3d00-c965-4b73-8141-3203a0637ae4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.886798 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.886447 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/580d3d00-c965-4b73-8141-3203a0637ae4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.886962 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.886924 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/580d3d00-c965-4b73-8141-3203a0637ae4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.886962 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.886947 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/580d3d00-c965-4b73-8141-3203a0637ae4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.887145 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.887096 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/580d3d00-c965-4b73-8141-3203a0637ae4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.888582 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.888558 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/580d3d00-c965-4b73-8141-3203a0637ae4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.889294 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.889271 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/580d3d00-c965-4b73-8141-3203a0637ae4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:57.896249 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:57.896086 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6zgg\" (UniqueName: \"kubernetes.io/projected/580d3d00-c965-4b73-8141-3203a0637ae4-kube-api-access-x6zgg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g75n7\" (UID: \"580d3d00-c965-4b73-8141-3203a0637ae4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:58.014863 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:58.014771 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:07:58.150666 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:58.150637 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7"] Jul 31 23:07:58.153151 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:07:58.153108 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod580d3d00_c965_4b73_8141_3203a0637ae4.slice/crio-da8eccd9d97136dad6c5ac2a962c1e88771de4403f62a2b6beac6f92a278b73a WatchSource:0}: Error finding container da8eccd9d97136dad6c5ac2a962c1e88771de4403f62a2b6beac6f92a278b73a: Status 404 returned error can't find the container with id da8eccd9d97136dad6c5ac2a962c1e88771de4403f62a2b6beac6f92a278b73a Jul 31 23:07:58.263440 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:58.263406 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" event={"ID":"580d3d00-c965-4b73-8141-3203a0637ae4","Type":"ContainerStarted","Data":"da8eccd9d97136dad6c5ac2a962c1e88771de4403f62a2b6beac6f92a278b73a"} Jul 31 23:07:59.270573 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:59.270531 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" event={"ID":"580d3d00-c965-4b73-8141-3203a0637ae4","Type":"ContainerStarted","Data":"6a3926c54a48d0c542c453333872966442575a339102c73c900c6076e5604df3"} Jul 31 23:07:59.289280 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:07:59.289224 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" podStartSLOduration=2.091438889 podStartE2EDuration="2.289207009s" totalImagesPullingTime="197.76812ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:07:59.28775636 +0000 UTC m=+635.760200544" watchObservedRunningTime="2026-07-31 23:07:59.289207009 +0000 UTC m=+635.761651191" Jul 31 23:08:08.015936 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:08.015892 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:08:08.016444 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:08.015999 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:08:08.029319 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:08.029287 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:08:08.328902 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:08.328872 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g75n7" Jul 31 23:08:14.391397 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.391362 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk"] Jul 31 23:08:14.395236 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.395217 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.397867 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.397841 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 31 23:08:14.402868 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.402843 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk"] Jul 31 23:08:14.425354 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.425311 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:08:14.553999 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.553964 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.554204 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.554006 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.554204 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.554026 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.554204 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.554143 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjpvp\" (UniqueName: \"kubernetes.io/projected/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-kube-api-access-vjpvp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.554204 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.554175 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.554204 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.554198 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.655557 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.655429 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.655557 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.655488 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.655557 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.655530 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.655838 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.655592 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjpvp\" (UniqueName: \"kubernetes.io/projected/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-kube-api-access-vjpvp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.655838 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.655612 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.655838 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.655636 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.656008 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.655916 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.656008 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.655994 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.656115 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.656090 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.657912 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.657888 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.658181 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.658159 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.663420 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.663399 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjpvp\" (UniqueName: \"kubernetes.io/projected/b9c7bbae-1bc8-4c7d-8f44-58bc318989d3-kube-api-access-vjpvp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk\" (UID: \"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.707086 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.707048 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:14.847872 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:14.847846 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk"] Jul 31 23:08:14.850053 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:08:14.850023 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9c7bbae_1bc8_4c7d_8f44_58bc318989d3.slice/crio-5a62b11c565d92f9bd479dbc21acd60cdc9510ebe72f98502434d37d2ab7d655 WatchSource:0}: Error finding container 5a62b11c565d92f9bd479dbc21acd60cdc9510ebe72f98502434d37d2ab7d655: Status 404 returned error can't find the container with id 5a62b11c565d92f9bd479dbc21acd60cdc9510ebe72f98502434d37d2ab7d655 Jul 31 23:08:15.355259 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:15.355225 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" event={"ID":"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3","Type":"ContainerStarted","Data":"708598fd020ee897cb4cfbd241c548455965c94320293d89e3fefca7e7b4e28a"} Jul 31 23:08:15.355475 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:15.355268 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" event={"ID":"b9c7bbae-1bc8-4c7d-8f44-58bc318989d3","Type":"ContainerStarted","Data":"5a62b11c565d92f9bd479dbc21acd60cdc9510ebe72f98502434d37d2ab7d655"} Jul 31 23:08:15.374006 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:15.373956 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" podStartSLOduration=1.172300194 podStartE2EDuration="1.373939111s" totalImagesPullingTime="201.638917ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:08:15.371633166 +0000 UTC m=+651.844077351" watchObservedRunningTime="2026-07-31 23:08:15.373939111 +0000 UTC m=+651.846383292" Jul 31 23:08:24.707943 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:24.707895 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:24.708406 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:24.707960 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:24.720877 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:24.720845 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:25.422035 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:25.422009 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk" Jul 31 23:08:28.366045 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:28.366007 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:08:28.429283 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:28.429251 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:08:48.213548 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:48.213493 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:08:48.271733 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:48.271694 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:08:49.983095 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:08:49.983059 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:09:09.001985 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.001943 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78b85c8676-k5qd9"] Jul 31 23:09:09.002428 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.002178 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-78b85c8676-k5qd9" podUID="acf8983c-ad7c-482d-8877-aebb71b6595e" containerName="authorino" containerID="cri-o://051f2c4d1152a024272e988623d1410ebf54c0b0492e3a04505285b25106e41a" gracePeriod=30 Jul 31 23:09:09.256823 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.256757 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78b85c8676-k5qd9" Jul 31 23:09:09.317380 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.317340 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/acf8983c-ad7c-482d-8877-aebb71b6595e-tls-cert\") pod \"acf8983c-ad7c-482d-8877-aebb71b6595e\" (UID: \"acf8983c-ad7c-482d-8877-aebb71b6595e\") " Jul 31 23:09:09.317576 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.317444 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5g4km\" (UniqueName: \"kubernetes.io/projected/acf8983c-ad7c-482d-8877-aebb71b6595e-kube-api-access-5g4km\") pod \"acf8983c-ad7c-482d-8877-aebb71b6595e\" (UID: \"acf8983c-ad7c-482d-8877-aebb71b6595e\") " Jul 31 23:09:09.319465 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.319433 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf8983c-ad7c-482d-8877-aebb71b6595e-kube-api-access-5g4km" (OuterVolumeSpecName: "kube-api-access-5g4km") pod "acf8983c-ad7c-482d-8877-aebb71b6595e" (UID: "acf8983c-ad7c-482d-8877-aebb71b6595e"). InnerVolumeSpecName "kube-api-access-5g4km". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:09:09.330494 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.330463 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acf8983c-ad7c-482d-8877-aebb71b6595e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "acf8983c-ad7c-482d-8877-aebb71b6595e" (UID: "acf8983c-ad7c-482d-8877-aebb71b6595e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 23:09:09.419032 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.418998 2561 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/acf8983c-ad7c-482d-8877-aebb71b6595e-tls-cert\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:09:09.419032 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.419026 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g4km\" (UniqueName: \"kubernetes.io/projected/acf8983c-ad7c-482d-8877-aebb71b6595e-kube-api-access-5g4km\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:09:09.489355 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.489323 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:09:09.593922 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.593893 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:09:09.647796 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.647759 2561 generic.go:356] "Generic (PLEG): container finished" podID="acf8983c-ad7c-482d-8877-aebb71b6595e" containerID="051f2c4d1152a024272e988623d1410ebf54c0b0492e3a04505285b25106e41a" exitCode=0 Jul 31 23:09:09.647796 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.647796 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78b85c8676-k5qd9" Jul 31 23:09:09.648023 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.647816 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78b85c8676-k5qd9" event={"ID":"acf8983c-ad7c-482d-8877-aebb71b6595e","Type":"ContainerDied","Data":"051f2c4d1152a024272e988623d1410ebf54c0b0492e3a04505285b25106e41a"} Jul 31 23:09:09.648023 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.647855 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78b85c8676-k5qd9" event={"ID":"acf8983c-ad7c-482d-8877-aebb71b6595e","Type":"ContainerDied","Data":"8272f7e935dfc64e9d5c4c4b37e8bfbf811dd4051f9ba760ca068ea6bf5f80fc"} Jul 31 23:09:09.648023 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.647877 2561 scope.go:117] "RemoveContainer" containerID="051f2c4d1152a024272e988623d1410ebf54c0b0492e3a04505285b25106e41a" Jul 31 23:09:09.657227 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.657204 2561 scope.go:117] "RemoveContainer" containerID="051f2c4d1152a024272e988623d1410ebf54c0b0492e3a04505285b25106e41a" Jul 31 23:09:09.657523 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:09:09.657485 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"051f2c4d1152a024272e988623d1410ebf54c0b0492e3a04505285b25106e41a\": container with ID starting with 051f2c4d1152a024272e988623d1410ebf54c0b0492e3a04505285b25106e41a not found: ID does not exist" containerID="051f2c4d1152a024272e988623d1410ebf54c0b0492e3a04505285b25106e41a" Jul 31 23:09:09.657625 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.657530 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"051f2c4d1152a024272e988623d1410ebf54c0b0492e3a04505285b25106e41a"} err="failed to get container status \"051f2c4d1152a024272e988623d1410ebf54c0b0492e3a04505285b25106e41a\": rpc error: code = NotFound desc = could not find container \"051f2c4d1152a024272e988623d1410ebf54c0b0492e3a04505285b25106e41a\": container with ID starting with 051f2c4d1152a024272e988623d1410ebf54c0b0492e3a04505285b25106e41a not found: ID does not exist" Jul 31 23:09:09.669174 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.669136 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78b85c8676-k5qd9"] Jul 31 23:09:09.673443 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:09.673417 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-78b85c8676-k5qd9"] Jul 31 23:09:10.132451 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:10.132410 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acf8983c-ad7c-482d-8877-aebb71b6595e" path="/var/lib/kubelet/pods/acf8983c-ad7c-482d-8877-aebb71b6595e/volumes" Jul 31 23:09:10.542934 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:10.542824 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:09:10.647416 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:10.647386 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:09:16.471879 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:16.471800 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:09:42.783377 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:09:42.783340 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:11:16.148612 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:11:16.148572 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:11:17.321699 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:11:17.321651 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:12:19.489439 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:12:19.489340 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:12:19.915294 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:12:19.915263 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:12:20.013891 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:12:20.013856 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:12:20.140257 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:12:20.140218 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:12:26.026279 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:12:26.026244 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:12:26.128013 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:12:26.127976 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:12:26.227022 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:12:26.226986 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:12:26.362901 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:12:26.362863 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:12:32.463418 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:12:32.463382 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:12:33.819176 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:12:33.819136 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:13:05.636326 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:13:05.636280 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:13:05.701318 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:13:05.701280 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:13:11.042797 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:13:11.042757 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:13:22.619651 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:13:22.619615 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:13:22.720946 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:13:22.720909 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:16:39.007764 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:16:39.007711 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:16:39.090398 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:16:39.090365 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:17:46.581380 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:46.581344 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg"] Jul 31 23:17:46.583989 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:46.581569 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" podUID="6b72cf24-2d85-4828-bdb5-16edeb5a8879" containerName="manager" containerID="cri-o://0758a13db3029b16bb7729832c34ec33aa77c8bc69d714aae62066654c54aade" gracePeriod=10 Jul 31 23:17:46.840712 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:46.840645 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" Jul 31 23:17:46.960752 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:46.960710 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b72cf24-2d85-4828-bdb5-16edeb5a8879-extensions-socket-volume\") pod \"6b72cf24-2d85-4828-bdb5-16edeb5a8879\" (UID: \"6b72cf24-2d85-4828-bdb5-16edeb5a8879\") " Jul 31 23:17:46.960752 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:46.960764 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpbt2\" (UniqueName: \"kubernetes.io/projected/6b72cf24-2d85-4828-bdb5-16edeb5a8879-kube-api-access-zpbt2\") pod \"6b72cf24-2d85-4828-bdb5-16edeb5a8879\" (UID: \"6b72cf24-2d85-4828-bdb5-16edeb5a8879\") " Jul 31 23:17:46.961200 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:46.961168 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b72cf24-2d85-4828-bdb5-16edeb5a8879-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6b72cf24-2d85-4828-bdb5-16edeb5a8879" (UID: "6b72cf24-2d85-4828-bdb5-16edeb5a8879"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:17:46.962960 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:46.962937 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b72cf24-2d85-4828-bdb5-16edeb5a8879-kube-api-access-zpbt2" (OuterVolumeSpecName: "kube-api-access-zpbt2") pod "6b72cf24-2d85-4828-bdb5-16edeb5a8879" (UID: "6b72cf24-2d85-4828-bdb5-16edeb5a8879"). InnerVolumeSpecName "kube-api-access-zpbt2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:17:47.061843 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:47.061804 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b72cf24-2d85-4828-bdb5-16edeb5a8879-extensions-socket-volume\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:17:47.061843 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:47.061836 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpbt2\" (UniqueName: \"kubernetes.io/projected/6b72cf24-2d85-4828-bdb5-16edeb5a8879-kube-api-access-zpbt2\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:17:47.324274 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:47.324242 2561 generic.go:356] "Generic (PLEG): container finished" podID="6b72cf24-2d85-4828-bdb5-16edeb5a8879" containerID="0758a13db3029b16bb7729832c34ec33aa77c8bc69d714aae62066654c54aade" exitCode=0 Jul 31 23:17:47.324462 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:47.324286 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" Jul 31 23:17:47.324462 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:47.324335 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" event={"ID":"6b72cf24-2d85-4828-bdb5-16edeb5a8879","Type":"ContainerDied","Data":"0758a13db3029b16bb7729832c34ec33aa77c8bc69d714aae62066654c54aade"} Jul 31 23:17:47.324462 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:47.324394 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg" event={"ID":"6b72cf24-2d85-4828-bdb5-16edeb5a8879","Type":"ContainerDied","Data":"7c1192a1182e6e0f6b63e802d3f99660bb97882b6cfc42b5688f29f933b8c7ab"} Jul 31 23:17:47.324462 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:47.324423 2561 scope.go:117] "RemoveContainer" containerID="0758a13db3029b16bb7729832c34ec33aa77c8bc69d714aae62066654c54aade" Jul 31 23:17:47.350081 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:47.350028 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg"] Jul 31 23:17:47.350540 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:47.350519 2561 scope.go:117] "RemoveContainer" containerID="0758a13db3029b16bb7729832c34ec33aa77c8bc69d714aae62066654c54aade" Jul 31 23:17:47.350911 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:17:47.350886 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0758a13db3029b16bb7729832c34ec33aa77c8bc69d714aae62066654c54aade\": container with ID starting with 0758a13db3029b16bb7729832c34ec33aa77c8bc69d714aae62066654c54aade not found: ID does not exist" containerID="0758a13db3029b16bb7729832c34ec33aa77c8bc69d714aae62066654c54aade" Jul 31 23:17:47.350995 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:47.350922 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0758a13db3029b16bb7729832c34ec33aa77c8bc69d714aae62066654c54aade"} err="failed to get container status \"0758a13db3029b16bb7729832c34ec33aa77c8bc69d714aae62066654c54aade\": rpc error: code = NotFound desc = could not find container \"0758a13db3029b16bb7729832c34ec33aa77c8bc69d714aae62066654c54aade\": container with ID starting with 0758a13db3029b16bb7729832c34ec33aa77c8bc69d714aae62066654c54aade not found: ID does not exist" Jul 31 23:17:47.353215 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:47.353192 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-55spg"] Jul 31 23:17:48.133417 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:17:48.133375 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b72cf24-2d85-4828-bdb5-16edeb5a8879" path="/var/lib/kubelet/pods/6b72cf24-2d85-4828-bdb5-16edeb5a8879/volumes" Jul 31 23:18:47.226516 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.226466 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm"] Jul 31 23:18:47.227008 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.226889 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b72cf24-2d85-4828-bdb5-16edeb5a8879" containerName="manager" Jul 31 23:18:47.227008 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.226901 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b72cf24-2d85-4828-bdb5-16edeb5a8879" containerName="manager" Jul 31 23:18:47.227008 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.226927 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf8983c-ad7c-482d-8877-aebb71b6595e" containerName="authorino" Jul 31 23:18:47.227008 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.226932 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf8983c-ad7c-482d-8877-aebb71b6595e" containerName="authorino" Jul 31 23:18:47.227008 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.226990 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b72cf24-2d85-4828-bdb5-16edeb5a8879" containerName="manager" Jul 31 23:18:47.227008 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.227003 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="acf8983c-ad7c-482d-8877-aebb71b6595e" containerName="authorino" Jul 31 23:18:47.231301 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.231274 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" Jul 31 23:18:47.234459 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.234431 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mcgdw\"" Jul 31 23:18:47.249051 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.249019 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm"] Jul 31 23:18:47.345321 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.345266 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb5mb\" (UniqueName: \"kubernetes.io/projected/ed49b28e-86a7-4c1f-8e54-1de6f33c54c0-kube-api-access-rb5mb\") pod \"kuadrant-operator-controller-manager-68cf589589-5wxlm\" (UID: \"ed49b28e-86a7-4c1f-8e54-1de6f33c54c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" Jul 31 23:18:47.345511 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.345385 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed49b28e-86a7-4c1f-8e54-1de6f33c54c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-5wxlm\" (UID: \"ed49b28e-86a7-4c1f-8e54-1de6f33c54c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" Jul 31 23:18:47.446672 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.446625 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb5mb\" (UniqueName: \"kubernetes.io/projected/ed49b28e-86a7-4c1f-8e54-1de6f33c54c0-kube-api-access-rb5mb\") pod \"kuadrant-operator-controller-manager-68cf589589-5wxlm\" (UID: \"ed49b28e-86a7-4c1f-8e54-1de6f33c54c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" Jul 31 23:18:47.446867 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.446799 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed49b28e-86a7-4c1f-8e54-1de6f33c54c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-5wxlm\" (UID: \"ed49b28e-86a7-4c1f-8e54-1de6f33c54c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" Jul 31 23:18:47.447183 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.447159 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed49b28e-86a7-4c1f-8e54-1de6f33c54c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-5wxlm\" (UID: \"ed49b28e-86a7-4c1f-8e54-1de6f33c54c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" Jul 31 23:18:47.456723 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.456683 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb5mb\" (UniqueName: \"kubernetes.io/projected/ed49b28e-86a7-4c1f-8e54-1de6f33c54c0-kube-api-access-rb5mb\") pod \"kuadrant-operator-controller-manager-68cf589589-5wxlm\" (UID: \"ed49b28e-86a7-4c1f-8e54-1de6f33c54c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" Jul 31 23:18:47.548648 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.548546 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" Jul 31 23:18:47.698818 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:47.698787 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm"] Jul 31 23:18:47.701156 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:18:47.701124 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded49b28e_86a7_4c1f_8e54_1de6f33c54c0.slice/crio-a849ced3fd498d1d94916e4b0b7e54f9df5f80cc23b722485c032732cf611584 WatchSource:0}: Error finding container a849ced3fd498d1d94916e4b0b7e54f9df5f80cc23b722485c032732cf611584: Status 404 returned error can't find the container with id a849ced3fd498d1d94916e4b0b7e54f9df5f80cc23b722485c032732cf611584 Jul 31 23:18:48.654468 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:48.654433 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" event={"ID":"ed49b28e-86a7-4c1f-8e54-1de6f33c54c0","Type":"ContainerStarted","Data":"e3e4bbcd050235b78e1e7ac356d2ec479ee19df046bda8aebd437bcb5df26a3a"} Jul 31 23:18:48.654896 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:48.654486 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" event={"ID":"ed49b28e-86a7-4c1f-8e54-1de6f33c54c0","Type":"ContainerStarted","Data":"a849ced3fd498d1d94916e4b0b7e54f9df5f80cc23b722485c032732cf611584"} Jul 31 23:18:48.654896 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:48.654536 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" Jul 31 23:18:48.677132 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:48.677079 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" podStartSLOduration=1.677063078 podStartE2EDuration="1.677063078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:18:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:18:48.675076599 +0000 UTC m=+1285.147520784" watchObservedRunningTime="2026-07-31 23:18:48.677063078 +0000 UTC m=+1285.149507261" Jul 31 23:18:59.661951 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:18:59.661917 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" Jul 31 23:19:34.452884 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:19:34.452848 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:19:48.666563 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:19:48.666457 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:21:22.139555 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:21:22.139436 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:21:59.035790 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:21:59.035752 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:22:15.596982 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.596939 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx"] Jul 31 23:22:15.601042 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.601022 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.603339 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.603317 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-a038c70d-kserve-self-signed-certs\"" Jul 31 23:22:15.610071 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.610048 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx"] Jul 31 23:22:15.618853 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.618829 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:22:15.662403 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.662369 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-home\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.662578 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.662410 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0fc7944-9b6a-472c-92dc-7fde588f7842-tls-certs\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.662578 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.662432 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfjxp\" (UniqueName: \"kubernetes.io/projected/b0fc7944-9b6a-472c-92dc-7fde588f7842-kube-api-access-rfjxp\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.662578 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.662493 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-dshm\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.662578 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.662541 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-tmp-dir\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.662578 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.662570 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-model-cache\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.763394 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.763321 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0fc7944-9b6a-472c-92dc-7fde588f7842-tls-certs\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.763700 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.763412 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfjxp\" (UniqueName: \"kubernetes.io/projected/b0fc7944-9b6a-472c-92dc-7fde588f7842-kube-api-access-rfjxp\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.763700 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.763465 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-dshm\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.763700 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.763488 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-tmp-dir\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.763700 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.763543 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-model-cache\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.763996 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.763758 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-home\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.764071 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.763997 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-model-cache\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.764071 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.764034 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-home\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.764353 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.764329 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-tmp-dir\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.765905 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.765874 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-dshm\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.766059 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.766041 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0fc7944-9b6a-472c-92dc-7fde588f7842-tls-certs\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.771127 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.771097 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfjxp\" (UniqueName: \"kubernetes.io/projected/b0fc7944-9b6a-472c-92dc-7fde588f7842-kube-api-access-rfjxp\") pod \"e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:15.912079 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:15.911982 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:16.053036 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:16.049286 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx"] Jul 31 23:22:16.757127 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:16.757094 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" event={"ID":"b0fc7944-9b6a-472c-92dc-7fde588f7842","Type":"ContainerStarted","Data":"29c23f624b5978b9168c939281397ce1264f9862def36e53a4b980d63f3ca35e"} Jul 31 23:22:16.757541 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:16.757136 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" event={"ID":"b0fc7944-9b6a-472c-92dc-7fde588f7842","Type":"ContainerStarted","Data":"f296f13beccb575a7c3a74d8513ebef3737a50709359cdf62a2b78a4c7d3b443"} Jul 31 23:22:16.774135 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:16.774076 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" podStartSLOduration=1.7740595529999998 podStartE2EDuration="1.774059553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:22:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:22:16.771868466 +0000 UTC m=+1493.244312657" watchObservedRunningTime="2026-07-31 23:22:16.774059553 +0000 UTC m=+1493.246503735" Jul 31 23:22:25.912592 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:25.912545 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:25.912592 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:25.912603 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:25.925484 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:25.925456 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:26.819742 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:26.819710 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:35.650135 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:35.650096 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:22:37.265624 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:37.265589 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx"] Jul 31 23:22:37.282334 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:37.282304 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:22:52.285876 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.285790 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" podUID="b0fc7944-9b6a-472c-92dc-7fde588f7842" containerName="main" containerID="cri-o://29c23f624b5978b9168c939281397ce1264f9862def36e53a4b980d63f3ca35e" gracePeriod=45 Jul 31 23:22:52.546985 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.546920 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:52.633726 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.633696 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-home\") pod \"b0fc7944-9b6a-472c-92dc-7fde588f7842\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " Jul 31 23:22:52.633726 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.633733 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-dshm\") pod \"b0fc7944-9b6a-472c-92dc-7fde588f7842\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " Jul 31 23:22:52.633974 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.633805 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0fc7944-9b6a-472c-92dc-7fde588f7842-tls-certs\") pod \"b0fc7944-9b6a-472c-92dc-7fde588f7842\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " Jul 31 23:22:52.633974 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.633839 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-tmp-dir\") pod \"b0fc7944-9b6a-472c-92dc-7fde588f7842\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " Jul 31 23:22:52.633974 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.633902 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfjxp\" (UniqueName: \"kubernetes.io/projected/b0fc7944-9b6a-472c-92dc-7fde588f7842-kube-api-access-rfjxp\") pod \"b0fc7944-9b6a-472c-92dc-7fde588f7842\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " Jul 31 23:22:52.634135 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.633996 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-model-cache\") pod \"b0fc7944-9b6a-472c-92dc-7fde588f7842\" (UID: \"b0fc7944-9b6a-472c-92dc-7fde588f7842\") " Jul 31 23:22:52.634135 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.634033 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-home" (OuterVolumeSpecName: "home") pod "b0fc7944-9b6a-472c-92dc-7fde588f7842" (UID: "b0fc7944-9b6a-472c-92dc-7fde588f7842"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:22:52.634135 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.634077 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b0fc7944-9b6a-472c-92dc-7fde588f7842" (UID: "b0fc7944-9b6a-472c-92dc-7fde588f7842"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:22:52.634263 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.634234 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-model-cache" (OuterVolumeSpecName: "model-cache") pod "b0fc7944-9b6a-472c-92dc-7fde588f7842" (UID: "b0fc7944-9b6a-472c-92dc-7fde588f7842"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:22:52.634341 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.634313 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-tmp-dir\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:22:52.634341 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.634339 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-model-cache\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:22:52.634341 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.634347 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-home\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:22:52.636406 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.636373 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0fc7944-9b6a-472c-92dc-7fde588f7842-kube-api-access-rfjxp" (OuterVolumeSpecName: "kube-api-access-rfjxp") pod "b0fc7944-9b6a-472c-92dc-7fde588f7842" (UID: "b0fc7944-9b6a-472c-92dc-7fde588f7842"). InnerVolumeSpecName "kube-api-access-rfjxp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:22:52.636538 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.636436 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-dshm" (OuterVolumeSpecName: "dshm") pod "b0fc7944-9b6a-472c-92dc-7fde588f7842" (UID: "b0fc7944-9b6a-472c-92dc-7fde588f7842"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:22:52.636538 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.636452 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0fc7944-9b6a-472c-92dc-7fde588f7842-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b0fc7944-9b6a-472c-92dc-7fde588f7842" (UID: "b0fc7944-9b6a-472c-92dc-7fde588f7842"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 23:22:52.735763 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.735720 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0fc7944-9b6a-472c-92dc-7fde588f7842-tls-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:22:52.735763 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.735761 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfjxp\" (UniqueName: \"kubernetes.io/projected/b0fc7944-9b6a-472c-92dc-7fde588f7842-kube-api-access-rfjxp\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:22:52.736036 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.735777 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0fc7944-9b6a-472c-92dc-7fde588f7842-dshm\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:22:52.942757 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.942717 2561 generic.go:356] "Generic (PLEG): container finished" podID="b0fc7944-9b6a-472c-92dc-7fde588f7842" containerID="29c23f624b5978b9168c939281397ce1264f9862def36e53a4b980d63f3ca35e" exitCode=0 Jul 31 23:22:52.942940 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.942769 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" Jul 31 23:22:52.942940 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.942803 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" event={"ID":"b0fc7944-9b6a-472c-92dc-7fde588f7842","Type":"ContainerDied","Data":"29c23f624b5978b9168c939281397ce1264f9862def36e53a4b980d63f3ca35e"} Jul 31 23:22:52.942940 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.942857 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx" event={"ID":"b0fc7944-9b6a-472c-92dc-7fde588f7842","Type":"ContainerDied","Data":"f296f13beccb575a7c3a74d8513ebef3737a50709359cdf62a2b78a4c7d3b443"} Jul 31 23:22:52.942940 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.942883 2561 scope.go:117] "RemoveContainer" containerID="29c23f624b5978b9168c939281397ce1264f9862def36e53a4b980d63f3ca35e" Jul 31 23:22:52.952858 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.952836 2561 scope.go:117] "RemoveContainer" containerID="29c23f624b5978b9168c939281397ce1264f9862def36e53a4b980d63f3ca35e" Jul 31 23:22:52.953170 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:22:52.953147 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29c23f624b5978b9168c939281397ce1264f9862def36e53a4b980d63f3ca35e\": container with ID starting with 29c23f624b5978b9168c939281397ce1264f9862def36e53a4b980d63f3ca35e not found: ID does not exist" containerID="29c23f624b5978b9168c939281397ce1264f9862def36e53a4b980d63f3ca35e" Jul 31 23:22:52.953230 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.953179 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c23f624b5978b9168c939281397ce1264f9862def36e53a4b980d63f3ca35e"} err="failed to get container status \"29c23f624b5978b9168c939281397ce1264f9862def36e53a4b980d63f3ca35e\": rpc error: code = NotFound desc = could not find container \"29c23f624b5978b9168c939281397ce1264f9862def36e53a4b980d63f3ca35e\": container with ID starting with 29c23f624b5978b9168c939281397ce1264f9862def36e53a4b980d63f3ca35e not found: ID does not exist" Jul 31 23:22:52.964471 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.964440 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx"] Jul 31 23:22:52.967229 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:52.967203 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-a038c70d-kserve-5b4c694575-lvxkx"] Jul 31 23:22:54.132943 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:22:54.132906 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0fc7944-9b6a-472c-92dc-7fde588f7842" path="/var/lib/kubelet/pods/b0fc7944-9b6a-472c-92dc-7fde588f7842/volumes" Jul 31 23:25:14.404389 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:25:14.404352 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:25:15.545823 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:25:15.545789 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:25:57.329057 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:25:57.329021 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:22.987301 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:22.987256 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:25.866964 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:25.866918 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:25.966050 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:25.965997 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:26.271433 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:26.271329 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:31.803390 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:31.803342 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:31.867364 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:31.867325 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:32.024885 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.024832 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:32.295253 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.295219 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv"] Jul 31 23:26:32.295692 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.295678 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0fc7944-9b6a-472c-92dc-7fde588f7842" containerName="main" Jul 31 23:26:32.295744 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.295693 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0fc7944-9b6a-472c-92dc-7fde588f7842" containerName="main" Jul 31 23:26:32.295797 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.295788 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0fc7944-9b6a-472c-92dc-7fde588f7842" containerName="main" Jul 31 23:26:32.298871 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.298853 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.301385 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.301369 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-fec77b77-openshift-default-dockercfg-qz7dw\"" Jul 31 23:26:32.313393 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.313360 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:32.318346 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.318316 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv"] Jul 31 23:26:32.455903 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.455864 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-credential-socket\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.456113 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.455977 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6632ffef-51ca-43ad-a778-d2bc27a20753-istiod-ca-cert\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.456113 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.456089 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-podinfo\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.456240 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.456138 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-data\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.456240 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.456179 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-workload-certs\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.456240 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.456207 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5m4h\" (UniqueName: \"kubernetes.io/projected/6632ffef-51ca-43ad-a778-d2bc27a20753-kube-api-access-n5m4h\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.456404 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.456270 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-token\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.456404 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.456326 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-workload-socket\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.456404 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.456356 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-envoy\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.546793 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.546691 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:32.557788 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.557752 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-data\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.557982 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.557793 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-workload-certs\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.557982 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.557819 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5m4h\" (UniqueName: \"kubernetes.io/projected/6632ffef-51ca-43ad-a778-d2bc27a20753-kube-api-access-n5m4h\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.557982 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.557838 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-token\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.557982 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.557866 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-workload-socket\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.557982 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.557889 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-envoy\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.557982 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.557930 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-credential-socket\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.558287 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.558005 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6632ffef-51ca-43ad-a778-d2bc27a20753-istiod-ca-cert\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.558287 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.558077 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-podinfo\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.558287 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.558232 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-data\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.558287 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.558235 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-workload-certs\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.558514 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.558475 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-workload-socket\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.558603 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.558579 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-credential-socket\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.558753 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.558735 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6632ffef-51ca-43ad-a778-d2bc27a20753-istiod-ca-cert\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.560391 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.560362 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-envoy\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.560694 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.560673 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-podinfo\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.565487 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.565457 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-token\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.565615 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.565543 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5m4h\" (UniqueName: \"kubernetes.io/projected/6632ffef-51ca-43ad-a778-d2bc27a20753-kube-api-access-n5m4h\") pod \"e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.609324 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.609286 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:32.751602 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.751577 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv"] Jul 31 23:26:32.754187 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:26:32.754155 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6632ffef_51ca_43ad_a778_d2bc27a20753.slice/crio-4bc45a4fa1f69014d0536ef33b96df2798020149de6adfe9a578c8451cc84b63 WatchSource:0}: Error finding container 4bc45a4fa1f69014d0536ef33b96df2798020149de6adfe9a578c8451cc84b63: Status 404 returned error can't find the container with id 4bc45a4fa1f69014d0536ef33b96df2798020149de6adfe9a578c8451cc84b63 Jul 31 23:26:32.756712 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.756676 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:26:32.756818 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.756746 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:26:32.756818 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:32.756780 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:26:33.094785 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:33.094684 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" event={"ID":"6632ffef-51ca-43ad-a778-d2bc27a20753","Type":"ContainerStarted","Data":"19ee651adbce4be6971cfc72f7897a61260f4c41cc05594a799dd34e6c7aacd7"} Jul 31 23:26:33.095178 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:33.094788 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" event={"ID":"6632ffef-51ca-43ad-a778-d2bc27a20753","Type":"ContainerStarted","Data":"4bc45a4fa1f69014d0536ef33b96df2798020149de6adfe9a578c8451cc84b63"} Jul 31 23:26:33.114003 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:33.113941 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" podStartSLOduration=1.11392379 podStartE2EDuration="1.11392379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:26:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:26:33.112567048 +0000 UTC m=+1749.585011232" watchObservedRunningTime="2026-07-31 23:26:33.11392379 +0000 UTC m=+1749.586367973" Jul 31 23:26:33.609810 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:33.609767 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:33.615004 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:33.614978 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:34.099962 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:34.099930 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:34.101136 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:34.101110 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:34.435735 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:34.435629 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:34.535158 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:34.535101 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:35.551235 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:35.551198 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:35.808812 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:35.808780 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:36.152723 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:36.152633 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:36.251288 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:36.251258 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:26:49.101518 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:49.101475 2561 patch_prober.go:28] interesting pod/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 31 23:26:49.101941 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:49.101547 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" podUID="6632ffef-51ca-43ad-a778-d2bc27a20753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 31 23:26:49.205676 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:49.205646 2561 generic.go:356] "Generic (PLEG): container finished" podID="6632ffef-51ca-43ad-a778-d2bc27a20753" containerID="19ee651adbce4be6971cfc72f7897a61260f4c41cc05594a799dd34e6c7aacd7" exitCode=0 Jul 31 23:26:49.205847 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:49.205726 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" event={"ID":"6632ffef-51ca-43ad-a778-d2bc27a20753","Type":"ContainerDied","Data":"19ee651adbce4be6971cfc72f7897a61260f4c41cc05594a799dd34e6c7aacd7"} Jul 31 23:26:49.206194 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:49.206177 2561 scope.go:117] "RemoveContainer" containerID="19ee651adbce4be6971cfc72f7897a61260f4c41cc05594a799dd34e6c7aacd7" Jul 31 23:26:49.207200 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:49.207170 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:26:49.207307 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:49.207248 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:26:49.207307 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:49.207285 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:26:50.213185 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:50.213146 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" event={"ID":"6632ffef-51ca-43ad-a778-d2bc27a20753","Type":"ContainerStarted","Data":"184b2d234f3524e954c684538a232f4f86ba42ad1144097b28ecdf20572147ae"} Jul 31 23:26:50.610342 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:50.610307 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:50.610548 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:50.610526 2561 patch_prober.go:28] interesting pod/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 31 23:26:50.610599 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:50.610573 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" podUID="6632ffef-51ca-43ad-a778-d2bc27a20753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 31 23:26:51.609873 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:51.609838 2561 patch_prober.go:28] interesting pod/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 31 23:26:51.610264 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:51.609907 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" podUID="6632ffef-51ca-43ad-a778-d2bc27a20753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 31 23:26:52.610296 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:52.610257 2561 patch_prober.go:28] interesting pod/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 31 23:26:52.610707 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:52.610325 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" podUID="6632ffef-51ca-43ad-a778-d2bc27a20753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 31 23:26:53.610045 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:53.610010 2561 patch_prober.go:28] interesting pod/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 31 23:26:53.610244 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:53.610065 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" podUID="6632ffef-51ca-43ad-a778-d2bc27a20753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 31 23:26:54.610655 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:54.610614 2561 patch_prober.go:28] interesting pod/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 31 23:26:54.611214 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:54.610713 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" podUID="6632ffef-51ca-43ad-a778-d2bc27a20753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 31 23:26:55.614374 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:55.614343 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:55.614883 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:55.614726 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:26:55.615546 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:26:55.615530 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:27:16.184473 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:16.184375 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:16.244246 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:16.244212 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:16.333064 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:16.333023 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:16.384697 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:16.384641 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:16.460466 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:16.460366 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv"] Jul 31 23:27:16.460692 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:16.460665 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" podUID="6632ffef-51ca-43ad-a778-d2bc27a20753" containerName="istio-proxy" containerID="cri-o://184b2d234f3524e954c684538a232f4f86ba42ad1144097b28ecdf20572147ae" gracePeriod=30 Jul 31 23:27:16.483928 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:16.483888 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:16.713178 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:16.713089 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:16.935490 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:16.935446 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:17.037477 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.037389 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:17.146925 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.146890 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:17.420717 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.420682 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq"] Jul 31 23:27:17.425051 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.425029 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.427665 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.427637 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-13204f82-gw-openshift-default-dockercfg-smwhl\"" Jul 31 23:27:17.437802 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.437771 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq"] Jul 31 23:27:17.443278 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.443190 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:17.525229 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.523068 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4xzp\" (UniqueName: \"kubernetes.io/projected/11250fc0-144a-4dd4-93e7-25d45d17f5e2-kube-api-access-f4xzp\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.525229 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.523156 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istiod-ca-cert\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.525229 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.523217 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-credential-socket\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.525229 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.523265 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-workload-certs\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.525229 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.523401 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-workload-socket\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.525229 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.523478 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-data\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.525229 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.523558 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-token\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.525229 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.523606 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-envoy\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.525229 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.523642 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-podinfo\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.624735 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.624694 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-workload-socket\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.624735 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.624731 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-data\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.624973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.624857 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-token\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.624973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.624902 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-envoy\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.624973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.624936 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-podinfo\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.625119 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.624983 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4xzp\" (UniqueName: \"kubernetes.io/projected/11250fc0-144a-4dd4-93e7-25d45d17f5e2-kube-api-access-f4xzp\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.625119 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.625041 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istiod-ca-cert\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.625119 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.625099 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-data\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.625119 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.625107 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-credential-socket\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.625317 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.625136 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-workload-socket\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.625317 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.625191 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-workload-certs\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.625427 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.625372 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-credential-socket\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.625610 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.625583 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-workload-certs\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.625735 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.625718 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istiod-ca-cert\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.627371 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.627340 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-envoy\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.627586 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.627570 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-podinfo\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.632373 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.632345 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-token\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.632373 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.632366 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4xzp\" (UniqueName: \"kubernetes.io/projected/11250fc0-144a-4dd4-93e7-25d45d17f5e2-kube-api-access-f4xzp\") pod \"e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.739234 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.739110 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:17.888711 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.888680 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq"] Jul 31 23:27:17.890489 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:27:17.890460 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11250fc0_144a_4dd4_93e7_25d45d17f5e2.slice/crio-22edad2ce0ff883b13da5ff4b6102fb1076ad0eaf775fb003f926cef329f1854 WatchSource:0}: Error finding container 22edad2ce0ff883b13da5ff4b6102fb1076ad0eaf775fb003f926cef329f1854: Status 404 returned error can't find the container with id 22edad2ce0ff883b13da5ff4b6102fb1076ad0eaf775fb003f926cef329f1854 Jul 31 23:27:17.892676 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.892647 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:27:17.892757 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.892711 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:27:17.892757 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.892743 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:27:17.895535 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:17.895495 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:18.370088 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:18.370048 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" event={"ID":"11250fc0-144a-4dd4-93e7-25d45d17f5e2","Type":"ContainerStarted","Data":"56c5af0bf1ed4cb66942918267aaa0d7fe0e13553289122d04f6aea215a87d96"} Jul 31 23:27:18.370281 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:18.370115 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" event={"ID":"11250fc0-144a-4dd4-93e7-25d45d17f5e2","Type":"ContainerStarted","Data":"22edad2ce0ff883b13da5ff4b6102fb1076ad0eaf775fb003f926cef329f1854"} Jul 31 23:27:18.390172 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:18.390117 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" podStartSLOduration=1.390097886 podStartE2EDuration="1.390097886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:27:18.3877306 +0000 UTC m=+1794.860174810" watchObservedRunningTime="2026-07-31 23:27:18.390097886 +0000 UTC m=+1794.862542070" Jul 31 23:27:18.740146 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:18.740052 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:18.745190 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:18.745160 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:19.375641 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:19.375612 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:19.376730 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:19.376708 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:19.831257 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:19.831220 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:19.941207 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:19.941170 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:20.113608 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:20.113463 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:20.430454 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:20.430358 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:20.533962 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:20.533927 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:21.726683 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.726656 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:27:21.773390 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.773304 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-podinfo\") pod \"6632ffef-51ca-43ad-a778-d2bc27a20753\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " Jul 31 23:27:21.773390 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.773342 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6632ffef-51ca-43ad-a778-d2bc27a20753-istiod-ca-cert\") pod \"6632ffef-51ca-43ad-a778-d2bc27a20753\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " Jul 31 23:27:21.773390 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.773368 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-workload-socket\") pod \"6632ffef-51ca-43ad-a778-d2bc27a20753\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " Jul 31 23:27:21.773390 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.773385 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-workload-certs\") pod \"6632ffef-51ca-43ad-a778-d2bc27a20753\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " Jul 31 23:27:21.773744 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.773410 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-token\") pod \"6632ffef-51ca-43ad-a778-d2bc27a20753\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " Jul 31 23:27:21.773744 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.773449 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-envoy\") pod \"6632ffef-51ca-43ad-a778-d2bc27a20753\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " Jul 31 23:27:21.773744 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.773540 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5m4h\" (UniqueName: \"kubernetes.io/projected/6632ffef-51ca-43ad-a778-d2bc27a20753-kube-api-access-n5m4h\") pod \"6632ffef-51ca-43ad-a778-d2bc27a20753\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " Jul 31 23:27:21.773744 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.773569 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-credential-socket\") pod \"6632ffef-51ca-43ad-a778-d2bc27a20753\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " Jul 31 23:27:21.773744 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.773612 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-data\") pod \"6632ffef-51ca-43ad-a778-d2bc27a20753\" (UID: \"6632ffef-51ca-43ad-a778-d2bc27a20753\") " Jul 31 23:27:21.774002 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.773779 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6632ffef-51ca-43ad-a778-d2bc27a20753" (UID: "6632ffef-51ca-43ad-a778-d2bc27a20753"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:27:21.774002 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.773876 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6632ffef-51ca-43ad-a778-d2bc27a20753-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6632ffef-51ca-43ad-a778-d2bc27a20753" (UID: "6632ffef-51ca-43ad-a778-d2bc27a20753"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:27:21.774002 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.773990 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6632ffef-51ca-43ad-a778-d2bc27a20753" (UID: "6632ffef-51ca-43ad-a778-d2bc27a20753"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:27:21.774166 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.774074 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6632ffef-51ca-43ad-a778-d2bc27a20753" (UID: "6632ffef-51ca-43ad-a778-d2bc27a20753"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:27:21.774166 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.774084 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-data" (OuterVolumeSpecName: "istio-data") pod "6632ffef-51ca-43ad-a778-d2bc27a20753" (UID: "6632ffef-51ca-43ad-a778-d2bc27a20753"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:27:21.774166 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.774133 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-credential-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:21.774166 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.774154 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6632ffef-51ca-43ad-a778-d2bc27a20753-istiod-ca-cert\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:21.774332 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.774168 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-workload-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:21.776018 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.775971 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6632ffef-51ca-43ad-a778-d2bc27a20753-kube-api-access-n5m4h" (OuterVolumeSpecName: "kube-api-access-n5m4h") pod "6632ffef-51ca-43ad-a778-d2bc27a20753" (UID: "6632ffef-51ca-43ad-a778-d2bc27a20753"). InnerVolumeSpecName "kube-api-access-n5m4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:27:21.776140 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.776037 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-token" (OuterVolumeSpecName: "istio-token") pod "6632ffef-51ca-43ad-a778-d2bc27a20753" (UID: "6632ffef-51ca-43ad-a778-d2bc27a20753"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:27:21.776140 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.776040 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6632ffef-51ca-43ad-a778-d2bc27a20753" (UID: "6632ffef-51ca-43ad-a778-d2bc27a20753"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 23:27:21.776366 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.776336 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6632ffef-51ca-43ad-a778-d2bc27a20753" (UID: "6632ffef-51ca-43ad-a778-d2bc27a20753"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:27:21.875013 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.874978 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-envoy\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:21.875013 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.875010 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5m4h\" (UniqueName: \"kubernetes.io/projected/6632ffef-51ca-43ad-a778-d2bc27a20753-kube-api-access-n5m4h\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:21.875013 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.875020 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-data\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:21.875284 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.875030 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-podinfo\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:21.875284 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.875039 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6632ffef-51ca-43ad-a778-d2bc27a20753-workload-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:21.875284 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:21.875047 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6632ffef-51ca-43ad-a778-d2bc27a20753-istio-token\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:22.395288 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:22.395191 2561 generic.go:356] "Generic (PLEG): container finished" podID="6632ffef-51ca-43ad-a778-d2bc27a20753" containerID="184b2d234f3524e954c684538a232f4f86ba42ad1144097b28ecdf20572147ae" exitCode=0 Jul 31 23:27:22.395288 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:22.395242 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" Jul 31 23:27:22.395288 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:22.395265 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" event={"ID":"6632ffef-51ca-43ad-a778-d2bc27a20753","Type":"ContainerDied","Data":"184b2d234f3524e954c684538a232f4f86ba42ad1144097b28ecdf20572147ae"} Jul 31 23:27:22.395597 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:22.395331 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv" event={"ID":"6632ffef-51ca-43ad-a778-d2bc27a20753","Type":"ContainerDied","Data":"4bc45a4fa1f69014d0536ef33b96df2798020149de6adfe9a578c8451cc84b63"} Jul 31 23:27:22.395597 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:22.395347 2561 scope.go:117] "RemoveContainer" containerID="184b2d234f3524e954c684538a232f4f86ba42ad1144097b28ecdf20572147ae" Jul 31 23:27:22.404890 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:22.404867 2561 scope.go:117] "RemoveContainer" containerID="19ee651adbce4be6971cfc72f7897a61260f4c41cc05594a799dd34e6c7aacd7" Jul 31 23:27:22.412975 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:22.412944 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv"] Jul 31 23:27:22.413984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:22.413956 2561 scope.go:117] "RemoveContainer" containerID="184b2d234f3524e954c684538a232f4f86ba42ad1144097b28ecdf20572147ae" Jul 31 23:27:22.414450 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:27:22.414428 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"184b2d234f3524e954c684538a232f4f86ba42ad1144097b28ecdf20572147ae\": container with ID starting with 184b2d234f3524e954c684538a232f4f86ba42ad1144097b28ecdf20572147ae not found: ID does not exist" containerID="184b2d234f3524e954c684538a232f4f86ba42ad1144097b28ecdf20572147ae" Jul 31 23:27:22.414571 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:22.414460 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"184b2d234f3524e954c684538a232f4f86ba42ad1144097b28ecdf20572147ae"} err="failed to get container status \"184b2d234f3524e954c684538a232f4f86ba42ad1144097b28ecdf20572147ae\": rpc error: code = NotFound desc = could not find container \"184b2d234f3524e954c684538a232f4f86ba42ad1144097b28ecdf20572147ae\": container with ID starting with 184b2d234f3524e954c684538a232f4f86ba42ad1144097b28ecdf20572147ae not found: ID does not exist" Jul 31 23:27:22.414571 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:22.414478 2561 scope.go:117] "RemoveContainer" containerID="19ee651adbce4be6971cfc72f7897a61260f4c41cc05594a799dd34e6c7aacd7" Jul 31 23:27:22.414874 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:27:22.414853 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19ee651adbce4be6971cfc72f7897a61260f4c41cc05594a799dd34e6c7aacd7\": container with ID starting with 19ee651adbce4be6971cfc72f7897a61260f4c41cc05594a799dd34e6c7aacd7 not found: ID does not exist" containerID="19ee651adbce4be6971cfc72f7897a61260f4c41cc05594a799dd34e6c7aacd7" Jul 31 23:27:22.414947 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:22.414881 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19ee651adbce4be6971cfc72f7897a61260f4c41cc05594a799dd34e6c7aacd7"} err="failed to get container status \"19ee651adbce4be6971cfc72f7897a61260f4c41cc05594a799dd34e6c7aacd7\": rpc error: code = NotFound desc = could not find container \"19ee651adbce4be6971cfc72f7897a61260f4c41cc05594a799dd34e6c7aacd7\": container with ID starting with 19ee651adbce4be6971cfc72f7897a61260f4c41cc05594a799dd34e6c7aacd7 not found: ID does not exist" Jul 31 23:27:22.416237 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:22.416214 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-fec77b77-openshift-default-f7c9c869b-np6xv"] Jul 31 23:27:24.134761 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:24.134726 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6632ffef-51ca-43ad-a778-d2bc27a20753" path="/var/lib/kubelet/pods/6632ffef-51ca-43ad-a778-d2bc27a20753/volumes" Jul 31 23:27:33.458921 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:33.458885 2561 generic.go:356] "Generic (PLEG): container finished" podID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" containerID="56c5af0bf1ed4cb66942918267aaa0d7fe0e13553289122d04f6aea215a87d96" exitCode=0 Jul 31 23:27:33.459483 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:33.458967 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" event={"ID":"11250fc0-144a-4dd4-93e7-25d45d17f5e2","Type":"ContainerDied","Data":"56c5af0bf1ed4cb66942918267aaa0d7fe0e13553289122d04f6aea215a87d96"} Jul 31 23:27:33.459606 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:33.459586 2561 scope.go:117] "RemoveContainer" containerID="56c5af0bf1ed4cb66942918267aaa0d7fe0e13553289122d04f6aea215a87d96" Jul 31 23:27:33.460790 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:33.460762 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:27:33.460864 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:33.460835 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:27:33.460901 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:33.460871 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:27:34.376396 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:34.376364 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:34.468309 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:34.468276 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" event={"ID":"11250fc0-144a-4dd4-93e7-25d45d17f5e2","Type":"ContainerStarted","Data":"8217be85fd345d3b572a9da5edd8f26677e9fd089256344972674244d581b16c"} Jul 31 23:27:34.740066 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:34.739962 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:34.741335 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:34.741303 2561 patch_prober.go:28] interesting pod/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 31 23:27:34.741491 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:34.741384 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" podUID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 31 23:27:35.739872 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:35.739829 2561 patch_prober.go:28] interesting pod/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 31 23:27:35.740380 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:35.739920 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" podUID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 31 23:27:36.740312 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:36.740272 2561 patch_prober.go:28] interesting pod/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 31 23:27:36.740836 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:36.740341 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" podUID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 31 23:27:37.740557 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:37.740488 2561 patch_prober.go:28] interesting pod/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 31 23:27:37.741075 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:37.740584 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" podUID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 31 23:27:38.740425 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:38.739901 2561 patch_prober.go:28] interesting pod/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 31 23:27:38.740425 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:38.739981 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" podUID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 31 23:27:39.744532 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:39.744487 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:39.744995 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:39.744973 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:39.745833 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:39.745811 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:50.422050 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:50.421996 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:50.480133 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:50.480094 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:50.538150 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:50.538116 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:50.639370 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:50.639331 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:50.700432 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:50.700324 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq"] Jul 31 23:27:50.700642 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:50.700610 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" podUID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" containerName="istio-proxy" containerID="cri-o://8217be85fd345d3b572a9da5edd8f26677e9fd089256344972674244d581b16c" gracePeriod=30 Jul 31 23:27:50.725810 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:50.725776 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:50.981473 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:50.981371 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:51.164367 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.164330 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:51.437815 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.437777 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h"] Jul 31 23:27:51.438263 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.438248 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6632ffef-51ca-43ad-a778-d2bc27a20753" containerName="istio-proxy" Jul 31 23:27:51.438311 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.438266 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6632ffef-51ca-43ad-a778-d2bc27a20753" containerName="istio-proxy" Jul 31 23:27:51.438311 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.438280 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6632ffef-51ca-43ad-a778-d2bc27a20753" containerName="istio-proxy" Jul 31 23:27:51.438311 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.438285 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6632ffef-51ca-43ad-a778-d2bc27a20753" containerName="istio-proxy" Jul 31 23:27:51.438416 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.438369 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6632ffef-51ca-43ad-a778-d2bc27a20753" containerName="istio-proxy" Jul 31 23:27:51.438416 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.438381 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6632ffef-51ca-43ad-a778-d2bc27a20753" containerName="istio-proxy" Jul 31 23:27:51.441879 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.441857 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.444879 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.444855 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-86538af5-openshift-default-dockercfg-9nmtp\"" Jul 31 23:27:51.455188 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.455156 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h"] Jul 31 23:27:51.468978 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.468942 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:51.493637 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.493599 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80ecf28a-7902-4201-aa05-9b727721bcbe-istiod-ca-cert\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.493829 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.493706 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-envoy\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.493829 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.493736 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-data\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.493829 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.493794 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-workload-socket\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.493829 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.493816 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-podinfo\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.494059 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.493911 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d4z9\" (UniqueName: \"kubernetes.io/projected/80ecf28a-7902-4201-aa05-9b727721bcbe-kube-api-access-9d4z9\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.494059 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.493956 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-credential-socket\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.494059 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.494017 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-token\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.494059 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.494054 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-workload-certs\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.564388 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.564352 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:51.594770 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.594725 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d4z9\" (UniqueName: \"kubernetes.io/projected/80ecf28a-7902-4201-aa05-9b727721bcbe-kube-api-access-9d4z9\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.594980 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.594779 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-credential-socket\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.594980 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.594831 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-token\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.594980 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.594864 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-workload-certs\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.594980 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.594935 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80ecf28a-7902-4201-aa05-9b727721bcbe-istiod-ca-cert\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.595215 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.594998 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-envoy\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.595215 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.595023 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-data\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.595215 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.595089 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-workload-socket\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.595215 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.595125 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-podinfo\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.595439 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.595298 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-credential-socket\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.595439 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.595341 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-workload-certs\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.595638 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.595617 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-workload-socket\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.595810 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.595783 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80ecf28a-7902-4201-aa05-9b727721bcbe-istiod-ca-cert\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.595956 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.595932 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-data\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.598198 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.598172 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-envoy\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.598548 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.598529 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-podinfo\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.607052 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.607026 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-token\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.607243 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.607220 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d4z9\" (UniqueName: \"kubernetes.io/projected/80ecf28a-7902-4201-aa05-9b727721bcbe-kube-api-access-9d4z9\") pod \"e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.755890 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.755790 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:51.898011 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.897984 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h"] Jul 31 23:27:51.899567 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:27:51.899529 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80ecf28a_7902_4201_aa05_9b727721bcbe.slice/crio-7809f528f2782fb2d3c733ab3ccb2b4ca40e067e7f18b298f051821677d38a80 WatchSource:0}: Error finding container 7809f528f2782fb2d3c733ab3ccb2b4ca40e067e7f18b298f051821677d38a80: Status 404 returned error can't find the container with id 7809f528f2782fb2d3c733ab3ccb2b4ca40e067e7f18b298f051821677d38a80 Jul 31 23:27:51.901836 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.901798 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:27:51.901949 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.901867 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:27:51.901949 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:51.901895 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:27:52.561228 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:52.561191 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" event={"ID":"80ecf28a-7902-4201-aa05-9b727721bcbe","Type":"ContainerStarted","Data":"ef314a3f767e951c8143edc14bfa6aeafe007e0c266f5da2f4b48a034b131ec8"} Jul 31 23:27:52.561765 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:52.561253 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" event={"ID":"80ecf28a-7902-4201-aa05-9b727721bcbe","Type":"ContainerStarted","Data":"7809f528f2782fb2d3c733ab3ccb2b4ca40e067e7f18b298f051821677d38a80"} Jul 31 23:27:52.583700 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:52.583640 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" podStartSLOduration=1.583622141 podStartE2EDuration="1.583622141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:27:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:27:52.581276082 +0000 UTC m=+1829.053720288" watchObservedRunningTime="2026-07-31 23:27:52.583622141 +0000 UTC m=+1829.056066325" Jul 31 23:27:52.756932 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:52.756898 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:52.762189 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:52.762159 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:53.567604 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:53.567575 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:53.568545 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:53.568524 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:27:53.913058 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:53.912969 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:54.008560 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:54.008522 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:54.747413 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:54.747366 2561 patch_prober.go:28] interesting pod/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 31 23:27:54.747930 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:54.747441 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" podUID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 31 23:27:55.020346 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:55.020244 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:55.326167 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:55.326119 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:55.569356 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:55.569319 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:55.667127 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:55.667027 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:27:55.963915 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:55.963888 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:56.043081 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.043036 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-token\") pod \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " Jul 31 23:27:56.043081 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.043081 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-credential-socket\") pod \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " Jul 31 23:27:56.043345 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.043106 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4xzp\" (UniqueName: \"kubernetes.io/projected/11250fc0-144a-4dd4-93e7-25d45d17f5e2-kube-api-access-f4xzp\") pod \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " Jul 31 23:27:56.043345 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.043123 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-workload-socket\") pod \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " Jul 31 23:27:56.043345 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.043314 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-data\") pod \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " Jul 31 23:27:56.043534 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.043408 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-workload-certs\") pod \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " Jul 31 23:27:56.043534 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.043434 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-envoy\") pod \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " Jul 31 23:27:56.043534 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.043466 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "11250fc0-144a-4dd4-93e7-25d45d17f5e2" (UID: "11250fc0-144a-4dd4-93e7-25d45d17f5e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:27:56.043534 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.043475 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istiod-ca-cert\") pod \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " Jul 31 23:27:56.043534 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.043484 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "11250fc0-144a-4dd4-93e7-25d45d17f5e2" (UID: "11250fc0-144a-4dd4-93e7-25d45d17f5e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:27:56.043791 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.043536 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-podinfo\") pod \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\" (UID: \"11250fc0-144a-4dd4-93e7-25d45d17f5e2\") " Jul 31 23:27:56.043791 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.043696 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "11250fc0-144a-4dd4-93e7-25d45d17f5e2" (UID: "11250fc0-144a-4dd4-93e7-25d45d17f5e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:27:56.043900 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.043836 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "11250fc0-144a-4dd4-93e7-25d45d17f5e2" (UID: "11250fc0-144a-4dd4-93e7-25d45d17f5e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:27:56.044054 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.044030 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "11250fc0-144a-4dd4-93e7-25d45d17f5e2" (UID: "11250fc0-144a-4dd4-93e7-25d45d17f5e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:27:56.044126 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.044055 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-workload-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:56.044126 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.044075 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istiod-ca-cert\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:56.044126 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.044087 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-credential-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:56.044126 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.044101 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-workload-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:56.045579 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.045553 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "11250fc0-144a-4dd4-93e7-25d45d17f5e2" (UID: "11250fc0-144a-4dd4-93e7-25d45d17f5e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:27:56.046079 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.046052 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "11250fc0-144a-4dd4-93e7-25d45d17f5e2" (UID: "11250fc0-144a-4dd4-93e7-25d45d17f5e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:27:56.046198 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.046060 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "11250fc0-144a-4dd4-93e7-25d45d17f5e2" (UID: "11250fc0-144a-4dd4-93e7-25d45d17f5e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 23:27:56.046256 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.046223 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11250fc0-144a-4dd4-93e7-25d45d17f5e2-kube-api-access-f4xzp" (OuterVolumeSpecName: "kube-api-access-f4xzp") pod "11250fc0-144a-4dd4-93e7-25d45d17f5e2" (UID: "11250fc0-144a-4dd4-93e7-25d45d17f5e2"). InnerVolumeSpecName "kube-api-access-f4xzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:27:56.145619 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.145592 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-token\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:56.145619 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.145617 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4xzp\" (UniqueName: \"kubernetes.io/projected/11250fc0-144a-4dd4-93e7-25d45d17f5e2-kube-api-access-f4xzp\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:56.145798 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.145629 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-data\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:56.145798 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.145638 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-envoy\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:56.145798 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.145645 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11250fc0-144a-4dd4-93e7-25d45d17f5e2-istio-podinfo\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:27:56.583564 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.583497 2561 generic.go:356] "Generic (PLEG): container finished" podID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" containerID="8217be85fd345d3b572a9da5edd8f26677e9fd089256344972674244d581b16c" exitCode=0 Jul 31 23:27:56.583776 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.583575 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" Jul 31 23:27:56.583776 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.583617 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" event={"ID":"11250fc0-144a-4dd4-93e7-25d45d17f5e2","Type":"ContainerDied","Data":"8217be85fd345d3b572a9da5edd8f26677e9fd089256344972674244d581b16c"} Jul 31 23:27:56.583776 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.583682 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq" event={"ID":"11250fc0-144a-4dd4-93e7-25d45d17f5e2","Type":"ContainerDied","Data":"22edad2ce0ff883b13da5ff4b6102fb1076ad0eaf775fb003f926cef329f1854"} Jul 31 23:27:56.583776 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.583716 2561 scope.go:117] "RemoveContainer" containerID="8217be85fd345d3b572a9da5edd8f26677e9fd089256344972674244d581b16c" Jul 31 23:27:56.592897 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.592873 2561 scope.go:117] "RemoveContainer" containerID="56c5af0bf1ed4cb66942918267aaa0d7fe0e13553289122d04f6aea215a87d96" Jul 31 23:27:56.601879 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.601855 2561 scope.go:117] "RemoveContainer" containerID="8217be85fd345d3b572a9da5edd8f26677e9fd089256344972674244d581b16c" Jul 31 23:27:56.602196 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:27:56.602174 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8217be85fd345d3b572a9da5edd8f26677e9fd089256344972674244d581b16c\": container with ID starting with 8217be85fd345d3b572a9da5edd8f26677e9fd089256344972674244d581b16c not found: ID does not exist" containerID="8217be85fd345d3b572a9da5edd8f26677e9fd089256344972674244d581b16c" Jul 31 23:27:56.602284 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.602211 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8217be85fd345d3b572a9da5edd8f26677e9fd089256344972674244d581b16c"} err="failed to get container status \"8217be85fd345d3b572a9da5edd8f26677e9fd089256344972674244d581b16c\": rpc error: code = NotFound desc = could not find container \"8217be85fd345d3b572a9da5edd8f26677e9fd089256344972674244d581b16c\": container with ID starting with 8217be85fd345d3b572a9da5edd8f26677e9fd089256344972674244d581b16c not found: ID does not exist" Jul 31 23:27:56.602284 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.602237 2561 scope.go:117] "RemoveContainer" containerID="56c5af0bf1ed4cb66942918267aaa0d7fe0e13553289122d04f6aea215a87d96" Jul 31 23:27:56.602481 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:27:56.602461 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56c5af0bf1ed4cb66942918267aaa0d7fe0e13553289122d04f6aea215a87d96\": container with ID starting with 56c5af0bf1ed4cb66942918267aaa0d7fe0e13553289122d04f6aea215a87d96 not found: ID does not exist" containerID="56c5af0bf1ed4cb66942918267aaa0d7fe0e13553289122d04f6aea215a87d96" Jul 31 23:27:56.602558 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.602488 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c5af0bf1ed4cb66942918267aaa0d7fe0e13553289122d04f6aea215a87d96"} err="failed to get container status \"56c5af0bf1ed4cb66942918267aaa0d7fe0e13553289122d04f6aea215a87d96\": rpc error: code = NotFound desc = could not find container \"56c5af0bf1ed4cb66942918267aaa0d7fe0e13553289122d04f6aea215a87d96\": container with ID starting with 56c5af0bf1ed4cb66942918267aaa0d7fe0e13553289122d04f6aea215a87d96 not found: ID does not exist" Jul 31 23:27:56.602636 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.602618 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq"] Jul 31 23:27:56.606051 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:56.606024 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-13204f82-gw-openshift-default-88fdd8f88-n25sq"] Jul 31 23:27:58.133463 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:27:58.133424 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" path="/var/lib/kubelet/pods/11250fc0-144a-4dd4-93e7-25d45d17f5e2/volumes" Jul 31 23:28:08.569199 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:08.569158 2561 patch_prober.go:28] interesting pod/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 31 23:28:08.569637 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:08.569239 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 31 23:28:08.651231 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:08.651195 2561 generic.go:356] "Generic (PLEG): container finished" podID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerID="ef314a3f767e951c8143edc14bfa6aeafe007e0c266f5da2f4b48a034b131ec8" exitCode=0 Jul 31 23:28:08.651390 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:08.651273 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" event={"ID":"80ecf28a-7902-4201-aa05-9b727721bcbe","Type":"ContainerDied","Data":"ef314a3f767e951c8143edc14bfa6aeafe007e0c266f5da2f4b48a034b131ec8"} Jul 31 23:28:08.651794 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:08.651776 2561 scope.go:117] "RemoveContainer" containerID="ef314a3f767e951c8143edc14bfa6aeafe007e0c266f5da2f4b48a034b131ec8" Jul 31 23:28:08.652801 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:08.652776 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:28:08.652859 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:08.652836 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:28:08.652906 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:08.652870 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:28:09.663053 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:09.663015 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" event={"ID":"80ecf28a-7902-4201-aa05-9b727721bcbe","Type":"ContainerStarted","Data":"4585ce33562f100910fc1e7a4ff82ddbe79a882bf9d7e2bb8f466a8363fe235b"} Jul 31 23:28:09.756632 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:09.756586 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:28:09.756846 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:09.756750 2561 patch_prober.go:28] interesting pod/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 31 23:28:09.756846 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:09.756797 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 31 23:28:10.756866 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:10.756821 2561 patch_prober.go:28] interesting pod/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 31 23:28:10.757493 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:10.757454 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 31 23:28:11.756946 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:11.756905 2561 patch_prober.go:28] interesting pod/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 31 23:28:11.757541 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:11.757484 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 31 23:28:12.756829 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:12.756783 2561 patch_prober.go:28] interesting pod/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 31 23:28:12.757060 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:12.756857 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 31 23:28:13.756587 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:13.756541 2561 patch_prober.go:28] interesting pod/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 31 23:28:13.756807 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:13.756609 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 31 23:28:14.756892 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:14.756844 2561 patch_prober.go:28] interesting pod/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 31 23:28:14.757424 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:14.756917 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 31 23:28:15.760670 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:15.760639 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:28:15.761128 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:15.760916 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:28:15.761786 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:15.761768 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:28:25.494520 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:25.494471 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:28:25.594086 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:25.594049 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:28:25.653277 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:25.653240 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:28:26.747836 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:26.747799 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h"] Jul 31 23:28:26.748295 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:26.748096 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerName="istio-proxy" containerID="cri-o://4585ce33562f100910fc1e7a4ff82ddbe79a882bf9d7e2bb8f466a8363fe235b" gracePeriod=30 Jul 31 23:28:26.764005 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:26.763969 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:28:26.985203 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:26.985167 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:28:27.237867 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.237823 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:28:27.469292 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.469252 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k"] Jul 31 23:28:27.469780 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.469763 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" containerName="istio-proxy" Jul 31 23:28:27.469835 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.469782 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" containerName="istio-proxy" Jul 31 23:28:27.469880 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.469868 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" containerName="istio-proxy" Jul 31 23:28:27.469930 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.469885 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" containerName="istio-proxy" Jul 31 23:28:27.472990 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.472969 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.475292 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.475267 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-7056468c-openshift-default-dockercfg-bgnw9\"" Jul 31 23:28:27.483341 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.483315 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k"] Jul 31 23:28:27.496205 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.496095 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:28:27.567886 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.567852 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4241c984-84fa-414c-b9af-7a99771f3bb5-istiod-ca-cert\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.568087 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.567900 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-credential-socket\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.568087 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.568020 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-workload-certs\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.568087 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.568061 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-data\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.568281 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.568094 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-podinfo\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.568281 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.568187 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-token\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.568281 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.568234 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtwzj\" (UniqueName: \"kubernetes.io/projected/4241c984-84fa-414c-b9af-7a99771f3bb5-kube-api-access-gtwzj\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.568414 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.568329 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-workload-socket\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.568414 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.568374 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-envoy\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.669783 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.669743 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-workload-socket\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.669783 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.669785 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-envoy\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.670066 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.669832 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4241c984-84fa-414c-b9af-7a99771f3bb5-istiod-ca-cert\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.670066 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.669855 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-credential-socket\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.670066 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.669889 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-workload-certs\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.670066 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.670007 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-data\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.670066 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.670053 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-podinfo\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.670323 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.670115 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-token\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.670323 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.670217 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-workload-socket\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.670323 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.670226 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-credential-socket\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.670323 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.670255 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtwzj\" (UniqueName: \"kubernetes.io/projected/4241c984-84fa-414c-b9af-7a99771f3bb5-kube-api-access-gtwzj\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.670485 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.670395 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-data\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.670611 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.670588 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-workload-certs\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.670712 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.670691 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4241c984-84fa-414c-b9af-7a99771f3bb5-istiod-ca-cert\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.671892 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.671864 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:28:27.672557 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.672533 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-envoy\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.672812 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.672796 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-podinfo\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.677809 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.677783 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtwzj\" (UniqueName: \"kubernetes.io/projected/4241c984-84fa-414c-b9af-7a99771f3bb5-kube-api-access-gtwzj\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.678289 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.678275 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-token\") pod \"e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.784567 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.784433 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:27.928835 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.928808 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k"] Jul 31 23:28:27.934946 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.934895 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:28:27.935063 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.934983 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:28:27.935063 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:27.935022 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:28:28.767730 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:28.767693 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" event={"ID":"4241c984-84fa-414c-b9af-7a99771f3bb5","Type":"ContainerStarted","Data":"97110edac73f0fa381709bf57f186069f178f497e65399bd276d927c378f74e4"} Jul 31 23:28:28.767920 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:28.767745 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" event={"ID":"4241c984-84fa-414c-b9af-7a99771f3bb5","Type":"ContainerStarted","Data":"1564959bd48ece3ce4a89744834a6ba0153a0ff1ea56b28651076cda01529551"} Jul 31 23:28:28.784990 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:28.784947 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:28.788512 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:28.788444 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" podStartSLOduration=1.788426982 podStartE2EDuration="1.788426982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:28:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:28:28.785868269 +0000 UTC m=+1865.258312458" watchObservedRunningTime="2026-07-31 23:28:28.788426982 +0000 UTC m=+1865.260871165" Jul 31 23:28:28.790293 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:28.790276 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:29.773094 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:29.773063 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:29.774096 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:29.774073 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:30.136959 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:30.136923 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:28:30.196230 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:30.196191 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:28:30.763884 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:30.763847 2561 patch_prober.go:28] interesting pod/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 31 23:28:30.764039 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:30.763907 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 31 23:28:31.230314 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:31.230277 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:28:31.501093 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:31.501011 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:28:31.704933 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:31.704897 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:28:31.807335 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:31.807304 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:28:32.017025 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.016957 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:28:32.121239 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.121201 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-envoy\") pod \"80ecf28a-7902-4201-aa05-9b727721bcbe\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " Jul 31 23:28:32.121437 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.121252 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-credential-socket\") pod \"80ecf28a-7902-4201-aa05-9b727721bcbe\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " Jul 31 23:28:32.121437 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.121272 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-token\") pod \"80ecf28a-7902-4201-aa05-9b727721bcbe\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " Jul 31 23:28:32.121437 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.121286 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-workload-certs\") pod \"80ecf28a-7902-4201-aa05-9b727721bcbe\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " Jul 31 23:28:32.121437 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.121347 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-workload-socket\") pod \"80ecf28a-7902-4201-aa05-9b727721bcbe\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " Jul 31 23:28:32.121437 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.121378 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-data\") pod \"80ecf28a-7902-4201-aa05-9b727721bcbe\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " Jul 31 23:28:32.121437 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.121402 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d4z9\" (UniqueName: \"kubernetes.io/projected/80ecf28a-7902-4201-aa05-9b727721bcbe-kube-api-access-9d4z9\") pod \"80ecf28a-7902-4201-aa05-9b727721bcbe\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " Jul 31 23:28:32.121812 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.121460 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80ecf28a-7902-4201-aa05-9b727721bcbe-istiod-ca-cert\") pod \"80ecf28a-7902-4201-aa05-9b727721bcbe\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " Jul 31 23:28:32.121812 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.121485 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-podinfo\") pod \"80ecf28a-7902-4201-aa05-9b727721bcbe\" (UID: \"80ecf28a-7902-4201-aa05-9b727721bcbe\") " Jul 31 23:28:32.121812 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.121581 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "80ecf28a-7902-4201-aa05-9b727721bcbe" (UID: "80ecf28a-7902-4201-aa05-9b727721bcbe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:28:32.121812 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.121699 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "80ecf28a-7902-4201-aa05-9b727721bcbe" (UID: "80ecf28a-7902-4201-aa05-9b727721bcbe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:28:32.122012 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.121873 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-credential-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:28:32.122012 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.121895 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-workload-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:28:32.122012 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.121919 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80ecf28a-7902-4201-aa05-9b727721bcbe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "80ecf28a-7902-4201-aa05-9b727721bcbe" (UID: "80ecf28a-7902-4201-aa05-9b727721bcbe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:28:32.122162 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.122001 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "80ecf28a-7902-4201-aa05-9b727721bcbe" (UID: "80ecf28a-7902-4201-aa05-9b727721bcbe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:28:32.122220 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.122185 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-data" (OuterVolumeSpecName: "istio-data") pod "80ecf28a-7902-4201-aa05-9b727721bcbe" (UID: "80ecf28a-7902-4201-aa05-9b727721bcbe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:28:32.123638 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.123613 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "80ecf28a-7902-4201-aa05-9b727721bcbe" (UID: "80ecf28a-7902-4201-aa05-9b727721bcbe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:28:32.123756 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.123721 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "80ecf28a-7902-4201-aa05-9b727721bcbe" (UID: "80ecf28a-7902-4201-aa05-9b727721bcbe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 23:28:32.124019 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.123999 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80ecf28a-7902-4201-aa05-9b727721bcbe-kube-api-access-9d4z9" (OuterVolumeSpecName: "kube-api-access-9d4z9") pod "80ecf28a-7902-4201-aa05-9b727721bcbe" (UID: "80ecf28a-7902-4201-aa05-9b727721bcbe"). InnerVolumeSpecName "kube-api-access-9d4z9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:28:32.124019 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.124008 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-token" (OuterVolumeSpecName: "istio-token") pod "80ecf28a-7902-4201-aa05-9b727721bcbe" (UID: "80ecf28a-7902-4201-aa05-9b727721bcbe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:28:32.223945 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.223906 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-data\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:28:32.223945 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.223944 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d4z9\" (UniqueName: \"kubernetes.io/projected/80ecf28a-7902-4201-aa05-9b727721bcbe-kube-api-access-9d4z9\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:28:32.224159 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.223963 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80ecf28a-7902-4201-aa05-9b727721bcbe-istiod-ca-cert\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:28:32.224159 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.223977 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-podinfo\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:28:32.224159 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.223989 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-envoy\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:28:32.224159 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.224002 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80ecf28a-7902-4201-aa05-9b727721bcbe-istio-token\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:28:32.224159 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.224015 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80ecf28a-7902-4201-aa05-9b727721bcbe-workload-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:28:32.791044 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.791010 2561 generic.go:356] "Generic (PLEG): container finished" podID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerID="4585ce33562f100910fc1e7a4ff82ddbe79a882bf9d7e2bb8f466a8363fe235b" exitCode=0 Jul 31 23:28:32.791526 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.791055 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" Jul 31 23:28:32.791526 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.791075 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" event={"ID":"80ecf28a-7902-4201-aa05-9b727721bcbe","Type":"ContainerDied","Data":"4585ce33562f100910fc1e7a4ff82ddbe79a882bf9d7e2bb8f466a8363fe235b"} Jul 31 23:28:32.791526 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.791119 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h" event={"ID":"80ecf28a-7902-4201-aa05-9b727721bcbe","Type":"ContainerDied","Data":"7809f528f2782fb2d3c733ab3ccb2b4ca40e067e7f18b298f051821677d38a80"} Jul 31 23:28:32.791526 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.791141 2561 scope.go:117] "RemoveContainer" containerID="4585ce33562f100910fc1e7a4ff82ddbe79a882bf9d7e2bb8f466a8363fe235b" Jul 31 23:28:32.800088 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.800071 2561 scope.go:117] "RemoveContainer" containerID="ef314a3f767e951c8143edc14bfa6aeafe007e0c266f5da2f4b48a034b131ec8" Jul 31 23:28:32.808002 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.807959 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h"] Jul 31 23:28:32.808789 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.808763 2561 scope.go:117] "RemoveContainer" containerID="4585ce33562f100910fc1e7a4ff82ddbe79a882bf9d7e2bb8f466a8363fe235b" Jul 31 23:28:32.809060 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:28:32.809040 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4585ce33562f100910fc1e7a4ff82ddbe79a882bf9d7e2bb8f466a8363fe235b\": container with ID starting with 4585ce33562f100910fc1e7a4ff82ddbe79a882bf9d7e2bb8f466a8363fe235b not found: ID does not exist" containerID="4585ce33562f100910fc1e7a4ff82ddbe79a882bf9d7e2bb8f466a8363fe235b" Jul 31 23:28:32.809125 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.809068 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4585ce33562f100910fc1e7a4ff82ddbe79a882bf9d7e2bb8f466a8363fe235b"} err="failed to get container status \"4585ce33562f100910fc1e7a4ff82ddbe79a882bf9d7e2bb8f466a8363fe235b\": rpc error: code = NotFound desc = could not find container \"4585ce33562f100910fc1e7a4ff82ddbe79a882bf9d7e2bb8f466a8363fe235b\": container with ID starting with 4585ce33562f100910fc1e7a4ff82ddbe79a882bf9d7e2bb8f466a8363fe235b not found: ID does not exist" Jul 31 23:28:32.809125 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.809086 2561 scope.go:117] "RemoveContainer" containerID="ef314a3f767e951c8143edc14bfa6aeafe007e0c266f5da2f4b48a034b131ec8" Jul 31 23:28:32.809327 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:28:32.809310 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef314a3f767e951c8143edc14bfa6aeafe007e0c266f5da2f4b48a034b131ec8\": container with ID starting with ef314a3f767e951c8143edc14bfa6aeafe007e0c266f5da2f4b48a034b131ec8 not found: ID does not exist" containerID="ef314a3f767e951c8143edc14bfa6aeafe007e0c266f5da2f4b48a034b131ec8" Jul 31 23:28:32.809384 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.809336 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef314a3f767e951c8143edc14bfa6aeafe007e0c266f5da2f4b48a034b131ec8"} err="failed to get container status \"ef314a3f767e951c8143edc14bfa6aeafe007e0c266f5da2f4b48a034b131ec8\": rpc error: code = NotFound desc = could not find container \"ef314a3f767e951c8143edc14bfa6aeafe007e0c266f5da2f4b48a034b131ec8\": container with ID starting with ef314a3f767e951c8143edc14bfa6aeafe007e0c266f5da2f4b48a034b131ec8 not found: ID does not exist" Jul 31 23:28:32.812511 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:32.812478 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-86538af5-openshift-default-7b4fddb7cb-hzm7h"] Jul 31 23:28:34.132604 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:34.132570 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" path="/var/lib/kubelet/pods/80ecf28a-7902-4201-aa05-9b727721bcbe/volumes" Jul 31 23:28:42.847574 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:42.847537 2561 generic.go:356] "Generic (PLEG): container finished" podID="4241c984-84fa-414c-b9af-7a99771f3bb5" containerID="97110edac73f0fa381709bf57f186069f178f497e65399bd276d927c378f74e4" exitCode=0 Jul 31 23:28:42.847995 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:42.847596 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" event={"ID":"4241c984-84fa-414c-b9af-7a99771f3bb5","Type":"ContainerDied","Data":"97110edac73f0fa381709bf57f186069f178f497e65399bd276d927c378f74e4"} Jul 31 23:28:42.848063 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:42.848049 2561 scope.go:117] "RemoveContainer" containerID="97110edac73f0fa381709bf57f186069f178f497e65399bd276d927c378f74e4" Jul 31 23:28:42.849144 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:42.849115 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:28:42.849214 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:42.849200 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:28:42.849250 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:42.849236 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:28:43.860098 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:43.860047 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" event={"ID":"4241c984-84fa-414c-b9af-7a99771f3bb5","Type":"ContainerStarted","Data":"63b847e6ebf232e8440963ccb752f85130a1b44993fef3b46304b92178b746d2"} Jul 31 23:28:44.773730 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:44.773686 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:44.784839 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:44.784815 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:44.786202 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:44.786177 2561 patch_prober.go:28] interesting pod/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 31 23:28:44.786298 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:44.786238 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" podUID="4241c984-84fa-414c-b9af-7a99771f3bb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 31 23:28:45.785622 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:45.785496 2561 patch_prober.go:28] interesting pod/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 31 23:28:45.785622 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:45.785596 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" podUID="4241c984-84fa-414c-b9af-7a99771f3bb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 31 23:28:46.784816 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:46.784777 2561 patch_prober.go:28] interesting pod/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 31 23:28:46.785011 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:46.784838 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" podUID="4241c984-84fa-414c-b9af-7a99771f3bb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 31 23:28:47.785229 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:47.785186 2561 patch_prober.go:28] interesting pod/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 31 23:28:47.785701 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:47.785255 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" podUID="4241c984-84fa-414c-b9af-7a99771f3bb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 31 23:28:48.784974 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:48.784927 2561 patch_prober.go:28] interesting pod/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 31 23:28:48.785174 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:48.785006 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" podUID="4241c984-84fa-414c-b9af-7a99771f3bb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 31 23:28:49.789440 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:49.789414 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:28:49.790460 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:28:49.790435 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:29:01.713820 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:01.713773 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:01.775878 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:01.775843 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:01.834288 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:01.834243 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:01.886787 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:01.886757 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:01.994724 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:01.994621 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k"] Jul 31 23:29:01.995249 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:01.995214 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" podUID="4241c984-84fa-414c-b9af-7a99771f3bb5" containerName="istio-proxy" containerID="cri-o://63b847e6ebf232e8440963ccb752f85130a1b44993fef3b46304b92178b746d2" gracePeriod=30 Jul 31 23:29:02.015236 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:02.015189 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:02.218860 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:02.218824 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:02.682476 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:02.682435 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:02.780463 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:02.780427 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:02.881002 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:02.880964 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:04.792295 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:04.792257 2561 patch_prober.go:28] interesting pod/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 31 23:29:04.792798 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:04.792319 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" podUID="4241c984-84fa-414c-b9af-7a99771f3bb5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 31 23:29:07.252481 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.252444 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:29:07.290480 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.290446 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-envoy\") pod \"4241c984-84fa-414c-b9af-7a99771f3bb5\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " Jul 31 23:29:07.290681 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.290548 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtwzj\" (UniqueName: \"kubernetes.io/projected/4241c984-84fa-414c-b9af-7a99771f3bb5-kube-api-access-gtwzj\") pod \"4241c984-84fa-414c-b9af-7a99771f3bb5\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " Jul 31 23:29:07.290681 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.290613 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-credential-socket\") pod \"4241c984-84fa-414c-b9af-7a99771f3bb5\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " Jul 31 23:29:07.290805 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.290785 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-podinfo\") pod \"4241c984-84fa-414c-b9af-7a99771f3bb5\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " Jul 31 23:29:07.290869 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.290829 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-token\") pod \"4241c984-84fa-414c-b9af-7a99771f3bb5\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " Jul 31 23:29:07.290869 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.290855 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-workload-socket\") pod \"4241c984-84fa-414c-b9af-7a99771f3bb5\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " Jul 31 23:29:07.290976 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.290880 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4241c984-84fa-414c-b9af-7a99771f3bb5" (UID: "4241c984-84fa-414c-b9af-7a99771f3bb5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:29:07.290976 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.290887 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4241c984-84fa-414c-b9af-7a99771f3bb5-istiod-ca-cert\") pod \"4241c984-84fa-414c-b9af-7a99771f3bb5\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " Jul 31 23:29:07.290976 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.290943 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-workload-certs\") pod \"4241c984-84fa-414c-b9af-7a99771f3bb5\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " Jul 31 23:29:07.291128 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.290998 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-data\") pod \"4241c984-84fa-414c-b9af-7a99771f3bb5\" (UID: \"4241c984-84fa-414c-b9af-7a99771f3bb5\") " Jul 31 23:29:07.291257 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.291215 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4241c984-84fa-414c-b9af-7a99771f3bb5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4241c984-84fa-414c-b9af-7a99771f3bb5" (UID: "4241c984-84fa-414c-b9af-7a99771f3bb5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:29:07.291257 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.291222 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4241c984-84fa-414c-b9af-7a99771f3bb5" (UID: "4241c984-84fa-414c-b9af-7a99771f3bb5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:29:07.291401 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.291293 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4241c984-84fa-414c-b9af-7a99771f3bb5" (UID: "4241c984-84fa-414c-b9af-7a99771f3bb5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:29:07.291471 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.291410 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-data" (OuterVolumeSpecName: "istio-data") pod "4241c984-84fa-414c-b9af-7a99771f3bb5" (UID: "4241c984-84fa-414c-b9af-7a99771f3bb5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:29:07.291618 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.291595 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-credential-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:29:07.291702 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.291624 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-workload-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:29:07.291702 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.291642 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4241c984-84fa-414c-b9af-7a99771f3bb5-istiod-ca-cert\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:29:07.291702 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.291655 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-workload-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:29:07.291702 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.291667 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-data\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:29:07.293392 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.293264 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4241c984-84fa-414c-b9af-7a99771f3bb5" (UID: "4241c984-84fa-414c-b9af-7a99771f3bb5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 23:29:07.293392 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.293347 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-token" (OuterVolumeSpecName: "istio-token") pod "4241c984-84fa-414c-b9af-7a99771f3bb5" (UID: "4241c984-84fa-414c-b9af-7a99771f3bb5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:29:07.293392 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.293350 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4241c984-84fa-414c-b9af-7a99771f3bb5-kube-api-access-gtwzj" (OuterVolumeSpecName: "kube-api-access-gtwzj") pod "4241c984-84fa-414c-b9af-7a99771f3bb5" (UID: "4241c984-84fa-414c-b9af-7a99771f3bb5"). InnerVolumeSpecName "kube-api-access-gtwzj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:29:07.293392 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.293349 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4241c984-84fa-414c-b9af-7a99771f3bb5" (UID: "4241c984-84fa-414c-b9af-7a99771f3bb5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:29:07.392768 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.392671 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtwzj\" (UniqueName: \"kubernetes.io/projected/4241c984-84fa-414c-b9af-7a99771f3bb5-kube-api-access-gtwzj\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:29:07.392768 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.392702 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-podinfo\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:29:07.392768 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.392712 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-token\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:29:07.392768 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.392720 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4241c984-84fa-414c-b9af-7a99771f3bb5-istio-envoy\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:29:07.988967 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.988930 2561 generic.go:356] "Generic (PLEG): container finished" podID="4241c984-84fa-414c-b9af-7a99771f3bb5" containerID="63b847e6ebf232e8440963ccb752f85130a1b44993fef3b46304b92178b746d2" exitCode=0 Jul 31 23:29:07.989131 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.988979 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" Jul 31 23:29:07.989131 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.989052 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" event={"ID":"4241c984-84fa-414c-b9af-7a99771f3bb5","Type":"ContainerDied","Data":"63b847e6ebf232e8440963ccb752f85130a1b44993fef3b46304b92178b746d2"} Jul 31 23:29:07.989131 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.989105 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k" event={"ID":"4241c984-84fa-414c-b9af-7a99771f3bb5","Type":"ContainerDied","Data":"1564959bd48ece3ce4a89744834a6ba0153a0ff1ea56b28651076cda01529551"} Jul 31 23:29:07.989320 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:07.989141 2561 scope.go:117] "RemoveContainer" containerID="63b847e6ebf232e8440963ccb752f85130a1b44993fef3b46304b92178b746d2" Jul 31 23:29:08.001805 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:08.001788 2561 scope.go:117] "RemoveContainer" containerID="97110edac73f0fa381709bf57f186069f178f497e65399bd276d927c378f74e4" Jul 31 23:29:08.012030 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:08.012008 2561 scope.go:117] "RemoveContainer" containerID="63b847e6ebf232e8440963ccb752f85130a1b44993fef3b46304b92178b746d2" Jul 31 23:29:08.012281 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:08.012259 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k"] Jul 31 23:29:08.012390 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:29:08.012370 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63b847e6ebf232e8440963ccb752f85130a1b44993fef3b46304b92178b746d2\": container with ID starting with 63b847e6ebf232e8440963ccb752f85130a1b44993fef3b46304b92178b746d2 not found: ID does not exist" containerID="63b847e6ebf232e8440963ccb752f85130a1b44993fef3b46304b92178b746d2" Jul 31 23:29:08.012436 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:08.012402 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63b847e6ebf232e8440963ccb752f85130a1b44993fef3b46304b92178b746d2"} err="failed to get container status \"63b847e6ebf232e8440963ccb752f85130a1b44993fef3b46304b92178b746d2\": rpc error: code = NotFound desc = could not find container \"63b847e6ebf232e8440963ccb752f85130a1b44993fef3b46304b92178b746d2\": container with ID starting with 63b847e6ebf232e8440963ccb752f85130a1b44993fef3b46304b92178b746d2 not found: ID does not exist" Jul 31 23:29:08.012436 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:08.012422 2561 scope.go:117] "RemoveContainer" containerID="97110edac73f0fa381709bf57f186069f178f497e65399bd276d927c378f74e4" Jul 31 23:29:08.012715 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:29:08.012691 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97110edac73f0fa381709bf57f186069f178f497e65399bd276d927c378f74e4\": container with ID starting with 97110edac73f0fa381709bf57f186069f178f497e65399bd276d927c378f74e4 not found: ID does not exist" containerID="97110edac73f0fa381709bf57f186069f178f497e65399bd276d927c378f74e4" Jul 31 23:29:08.012812 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:08.012715 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97110edac73f0fa381709bf57f186069f178f497e65399bd276d927c378f74e4"} err="failed to get container status \"97110edac73f0fa381709bf57f186069f178f497e65399bd276d927c378f74e4\": rpc error: code = NotFound desc = could not find container \"97110edac73f0fa381709bf57f186069f178f497e65399bd276d927c378f74e4\": container with ID starting with 97110edac73f0fa381709bf57f186069f178f497e65399bd276d927c378f74e4 not found: ID does not exist" Jul 31 23:29:08.023371 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:08.023338 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-7056468c-openshift-default-7f7798489b-l9z8k"] Jul 31 23:29:08.134232 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:08.134187 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4241c984-84fa-414c-b9af-7a99771f3bb5" path="/var/lib/kubelet/pods/4241c984-84fa-414c-b9af-7a99771f3bb5/volumes" Jul 31 23:29:14.484824 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:14.484787 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:20.639221 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:20.639176 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:20.739995 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:20.739961 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:20.842610 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:20.842571 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:20.940447 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:20.940343 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:21.334135 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:21.334104 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:36.482998 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:36.482967 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:47.278553 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:47.278491 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:53.070491 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:53.070451 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:53.173966 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:53.173930 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:29:53.271364 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:29:53.271331 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:03.016175 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:03.016130 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:09.116406 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:09.116367 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:14.792299 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:14.792264 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:14.892516 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:14.892456 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:14.989236 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:14.989202 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:25.487141 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:25.487030 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:36.451160 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:36.451117 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:42.282571 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:42.282526 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:42.335767 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:42.335720 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:42.447161 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:42.447099 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:42.574968 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:42.574930 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:43.167170 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:43.167127 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:43.265659 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:43.265612 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:43.365483 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:43.365442 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:43.649382 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:43.649347 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:43.749727 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:43.749691 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:44.770976 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:44.770939 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:50.450841 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:50.450796 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:50.552260 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:50.552210 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:51.105231 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:51.105190 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:57.290406 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.290363 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:57.388342 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.388302 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:57.520760 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.520727 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:57.812257 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.812223 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn"] Jul 31 23:30:57.812678 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.812664 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerName="istio-proxy" Jul 31 23:30:57.812736 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.812680 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerName="istio-proxy" Jul 31 23:30:57.812736 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.812699 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4241c984-84fa-414c-b9af-7a99771f3bb5" containerName="istio-proxy" Jul 31 23:30:57.812736 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.812705 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4241c984-84fa-414c-b9af-7a99771f3bb5" containerName="istio-proxy" Jul 31 23:30:57.812736 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.812716 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" containerName="istio-proxy" Jul 31 23:30:57.812736 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.812721 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="11250fc0-144a-4dd4-93e7-25d45d17f5e2" containerName="istio-proxy" Jul 31 23:30:57.812736 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.812729 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4241c984-84fa-414c-b9af-7a99771f3bb5" containerName="istio-proxy" Jul 31 23:30:57.812736 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.812734 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4241c984-84fa-414c-b9af-7a99771f3bb5" containerName="istio-proxy" Jul 31 23:30:57.812950 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.812802 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4241c984-84fa-414c-b9af-7a99771f3bb5" containerName="istio-proxy" Jul 31 23:30:57.812950 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.812809 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerName="istio-proxy" Jul 31 23:30:57.812950 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.812816 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerName="istio-proxy" Jul 31 23:30:57.812950 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.812827 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4241c984-84fa-414c-b9af-7a99771f3bb5" containerName="istio-proxy" Jul 31 23:30:57.815707 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.815691 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:57.818425 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.818407 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-124aaef2-openshift-default-dockercfg-9pqpq\"" Jul 31 23:30:57.831631 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.831601 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn"] Jul 31 23:30:57.837474 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.837442 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:57.899516 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.899429 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-workload-certs\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:57.899984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.899526 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434d4206-fc4b-4154-b4a1-de24d84e16ab-istiod-ca-cert\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:57.899984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.899583 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-workload-socket\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:57.899984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.899614 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-credential-socket\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:57.899984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.899660 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-podinfo\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:57.899984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.899704 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7588\" (UniqueName: \"kubernetes.io/projected/434d4206-fc4b-4154-b4a1-de24d84e16ab-kube-api-access-b7588\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:57.899984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.899739 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-token\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:57.899984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.899772 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-envoy\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:57.899984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.899855 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-data\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:57.979464 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:57.979417 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:30:58.001308 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.001273 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-data\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.001493 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.001319 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-workload-certs\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.001493 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.001446 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434d4206-fc4b-4154-b4a1-de24d84e16ab-istiod-ca-cert\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.001662 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.001556 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-workload-socket\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.001662 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.001584 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-credential-socket\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.001662 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.001629 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-podinfo\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.001828 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.001675 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7588\" (UniqueName: \"kubernetes.io/projected/434d4206-fc4b-4154-b4a1-de24d84e16ab-kube-api-access-b7588\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.001828 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.001706 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-token\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.001828 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.001741 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-envoy\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.001828 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.001803 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-data\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.001828 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.001705 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-workload-certs\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.002076 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.002043 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-workload-socket\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.002181 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.002155 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-credential-socket\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.002388 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.002368 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434d4206-fc4b-4154-b4a1-de24d84e16ab-istiod-ca-cert\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.004278 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.004256 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-envoy\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.004441 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.004425 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-podinfo\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.009360 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.009332 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-token\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.009496 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.009478 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7588\" (UniqueName: \"kubernetes.io/projected/434d4206-fc4b-4154-b4a1-de24d84e16ab-kube-api-access-b7588\") pod \"e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.126414 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.126313 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:58.276632 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.276606 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn"] Jul 31 23:30:58.278783 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:30:58.278754 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod434d4206_fc4b_4154_b4a1_de24d84e16ab.slice/crio-3d7a11360e28263217a86e8c7232f5d6024ff537d0f695ceff341ed6168c5a6f WatchSource:0}: Error finding container 3d7a11360e28263217a86e8c7232f5d6024ff537d0f695ceff341ed6168c5a6f: Status 404 returned error can't find the container with id 3d7a11360e28263217a86e8c7232f5d6024ff537d0f695ceff341ed6168c5a6f Jul 31 23:30:58.281039 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.281007 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:30:58.281146 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.281084 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:30:58.281146 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.281132 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:30:58.588212 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.588178 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" event={"ID":"434d4206-fc4b-4154-b4a1-de24d84e16ab","Type":"ContainerStarted","Data":"3554dd0fa8f9a2f641a1459df74506ad44ce5a1b7b23dcd121c70a921792b947"} Jul 31 23:30:58.588632 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.588232 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" event={"ID":"434d4206-fc4b-4154-b4a1-de24d84e16ab","Type":"ContainerStarted","Data":"3d7a11360e28263217a86e8c7232f5d6024ff537d0f695ceff341ed6168c5a6f"} Jul 31 23:30:58.610934 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:58.610886 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" podStartSLOduration=1.610868124 podStartE2EDuration="1.610868124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:30:58.608355045 +0000 UTC m=+2015.080799228" watchObservedRunningTime="2026-07-31 23:30:58.610868124 +0000 UTC m=+2015.083312308" Jul 31 23:30:59.126812 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:59.126775 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:59.131625 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:59.131598 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:59.593682 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:59.593645 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:30:59.594632 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:30:59.594612 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:31:00.480513 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:00.480472 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:00.580143 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:00.580101 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:01.589438 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:01.589401 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:01.828191 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:01.828154 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:02.165827 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:02.165791 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:02.264465 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:02.264420 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:14.594891 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:14.594852 2561 patch_prober.go:28] interesting pod/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 31 23:31:14.595407 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:14.594926 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" podUID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 31 23:31:14.696126 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:14.696089 2561 generic.go:356] "Generic (PLEG): container finished" podID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerID="3554dd0fa8f9a2f641a1459df74506ad44ce5a1b7b23dcd121c70a921792b947" exitCode=0 Jul 31 23:31:14.696303 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:14.696163 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" event={"ID":"434d4206-fc4b-4154-b4a1-de24d84e16ab","Type":"ContainerDied","Data":"3554dd0fa8f9a2f641a1459df74506ad44ce5a1b7b23dcd121c70a921792b947"} Jul 31 23:31:14.696624 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:14.696609 2561 scope.go:117] "RemoveContainer" containerID="3554dd0fa8f9a2f641a1459df74506ad44ce5a1b7b23dcd121c70a921792b947" Jul 31 23:31:14.697678 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:14.697654 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:31:14.697741 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:14.697723 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:31:14.697777 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:14.697757 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:31:15.703404 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:15.703369 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" event={"ID":"434d4206-fc4b-4154-b4a1-de24d84e16ab","Type":"ContainerStarted","Data":"e3ce7644f5b2a047cad03eec86202ee35d4c219307fa7fca31d808e12e65d07b"} Jul 31 23:31:16.127192 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:16.127153 2561 patch_prober.go:28] interesting pod/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 31 23:31:16.127380 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:16.127235 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" podUID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 31 23:31:16.132625 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:16.132600 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:31:17.126919 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:17.126877 2561 patch_prober.go:28] interesting pod/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 31 23:31:17.127341 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:17.126946 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" podUID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 31 23:31:18.127001 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:18.126962 2561 patch_prober.go:28] interesting pod/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 31 23:31:18.127531 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:18.127042 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" podUID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 31 23:31:19.126862 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:19.126826 2561 patch_prober.go:28] interesting pod/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 31 23:31:19.127036 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:19.126883 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" podUID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 31 23:31:20.127394 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:20.127354 2561 patch_prober.go:28] interesting pod/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 31 23:31:20.127815 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:20.127430 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" podUID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 31 23:31:21.130610 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:21.130581 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:31:21.130985 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:21.130917 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:31:21.131665 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:21.131646 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:31:25.447252 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:25.447213 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:26.081166 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:26.081122 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:26.181299 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:26.181256 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:26.483699 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:26.483576 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:32.212343 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:32.212307 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:32.313283 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:32.313246 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:43.422968 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:43.422899 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:43.523380 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:43.523343 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:43.579305 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:43.579265 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:43.948665 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:43.948615 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:49.660677 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:49.660583 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:49.760521 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:49.760475 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:49.893117 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:49.893081 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn"] Jul 31 23:31:49.893360 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:49.893334 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" podUID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerName="istio-proxy" containerID="cri-o://e3ce7644f5b2a047cad03eec86202ee35d4c219307fa7fca31d808e12e65d07b" gracePeriod=30 Jul 31 23:31:49.913068 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:49.912959 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:50.185046 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:50.184961 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:50.282476 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:50.282438 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:50.384492 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:50.384455 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:50.481785 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:50.481679 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:51.133979 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:51.133937 2561 patch_prober.go:28] interesting pod/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 31 23:31:51.134449 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:51.134021 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" podUID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 31 23:31:55.145344 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.145313 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:31:55.280972 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.280879 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-token\") pod \"434d4206-fc4b-4154-b4a1-de24d84e16ab\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " Jul 31 23:31:55.281142 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.280984 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-envoy\") pod \"434d4206-fc4b-4154-b4a1-de24d84e16ab\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " Jul 31 23:31:55.281142 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281021 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-podinfo\") pod \"434d4206-fc4b-4154-b4a1-de24d84e16ab\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " Jul 31 23:31:55.281142 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281050 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434d4206-fc4b-4154-b4a1-de24d84e16ab-istiod-ca-cert\") pod \"434d4206-fc4b-4154-b4a1-de24d84e16ab\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " Jul 31 23:31:55.281142 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281080 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-credential-socket\") pod \"434d4206-fc4b-4154-b4a1-de24d84e16ab\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " Jul 31 23:31:55.281142 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281106 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-data\") pod \"434d4206-fc4b-4154-b4a1-de24d84e16ab\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " Jul 31 23:31:55.281415 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281234 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-workload-socket\") pod \"434d4206-fc4b-4154-b4a1-de24d84e16ab\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " Jul 31 23:31:55.281415 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281305 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-workload-certs\") pod \"434d4206-fc4b-4154-b4a1-de24d84e16ab\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " Jul 31 23:31:55.281415 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281357 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7588\" (UniqueName: \"kubernetes.io/projected/434d4206-fc4b-4154-b4a1-de24d84e16ab-kube-api-access-b7588\") pod \"434d4206-fc4b-4154-b4a1-de24d84e16ab\" (UID: \"434d4206-fc4b-4154-b4a1-de24d84e16ab\") " Jul 31 23:31:55.281415 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281400 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-data" (OuterVolumeSpecName: "istio-data") pod "434d4206-fc4b-4154-b4a1-de24d84e16ab" (UID: "434d4206-fc4b-4154-b4a1-de24d84e16ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:31:55.281662 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281395 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "434d4206-fc4b-4154-b4a1-de24d84e16ab" (UID: "434d4206-fc4b-4154-b4a1-de24d84e16ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:31:55.281662 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281542 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/434d4206-fc4b-4154-b4a1-de24d84e16ab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "434d4206-fc4b-4154-b4a1-de24d84e16ab" (UID: "434d4206-fc4b-4154-b4a1-de24d84e16ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:31:55.281662 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281640 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "434d4206-fc4b-4154-b4a1-de24d84e16ab" (UID: "434d4206-fc4b-4154-b4a1-de24d84e16ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:31:55.281820 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281669 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "434d4206-fc4b-4154-b4a1-de24d84e16ab" (UID: "434d4206-fc4b-4154-b4a1-de24d84e16ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:31:55.281876 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281839 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434d4206-fc4b-4154-b4a1-de24d84e16ab-istiod-ca-cert\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:31:55.281876 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281862 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-credential-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:31:55.281986 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281876 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-data\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:31:55.281986 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281888 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-workload-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:31:55.281986 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.281900 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-workload-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:31:55.283385 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.283360 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "434d4206-fc4b-4154-b4a1-de24d84e16ab" (UID: "434d4206-fc4b-4154-b4a1-de24d84e16ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:31:55.283662 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.283628 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/434d4206-fc4b-4154-b4a1-de24d84e16ab-kube-api-access-b7588" (OuterVolumeSpecName: "kube-api-access-b7588") pod "434d4206-fc4b-4154-b4a1-de24d84e16ab" (UID: "434d4206-fc4b-4154-b4a1-de24d84e16ab"). InnerVolumeSpecName "kube-api-access-b7588". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:31:55.283763 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.283687 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "434d4206-fc4b-4154-b4a1-de24d84e16ab" (UID: "434d4206-fc4b-4154-b4a1-de24d84e16ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 23:31:55.283763 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.283721 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-token" (OuterVolumeSpecName: "istio-token") pod "434d4206-fc4b-4154-b4a1-de24d84e16ab" (UID: "434d4206-fc4b-4154-b4a1-de24d84e16ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:31:55.382983 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.382944 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7588\" (UniqueName: \"kubernetes.io/projected/434d4206-fc4b-4154-b4a1-de24d84e16ab-kube-api-access-b7588\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:31:55.382983 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.382978 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-token\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:31:55.382983 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.382987 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-envoy\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:31:55.382983 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.382995 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434d4206-fc4b-4154-b4a1-de24d84e16ab-istio-podinfo\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:31:55.916221 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.916185 2561 generic.go:356] "Generic (PLEG): container finished" podID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerID="e3ce7644f5b2a047cad03eec86202ee35d4c219307fa7fca31d808e12e65d07b" exitCode=0 Jul 31 23:31:55.916429 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.916243 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" Jul 31 23:31:55.916429 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.916272 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" event={"ID":"434d4206-fc4b-4154-b4a1-de24d84e16ab","Type":"ContainerDied","Data":"e3ce7644f5b2a047cad03eec86202ee35d4c219307fa7fca31d808e12e65d07b"} Jul 31 23:31:55.916429 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.916329 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn" event={"ID":"434d4206-fc4b-4154-b4a1-de24d84e16ab","Type":"ContainerDied","Data":"3d7a11360e28263217a86e8c7232f5d6024ff537d0f695ceff341ed6168c5a6f"} Jul 31 23:31:55.916429 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.916353 2561 scope.go:117] "RemoveContainer" containerID="e3ce7644f5b2a047cad03eec86202ee35d4c219307fa7fca31d808e12e65d07b" Jul 31 23:31:55.925942 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.925920 2561 scope.go:117] "RemoveContainer" containerID="3554dd0fa8f9a2f641a1459df74506ad44ce5a1b7b23dcd121c70a921792b947" Jul 31 23:31:55.934538 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.934513 2561 scope.go:117] "RemoveContainer" containerID="e3ce7644f5b2a047cad03eec86202ee35d4c219307fa7fca31d808e12e65d07b" Jul 31 23:31:55.934820 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:31:55.934794 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3ce7644f5b2a047cad03eec86202ee35d4c219307fa7fca31d808e12e65d07b\": container with ID starting with e3ce7644f5b2a047cad03eec86202ee35d4c219307fa7fca31d808e12e65d07b not found: ID does not exist" containerID="e3ce7644f5b2a047cad03eec86202ee35d4c219307fa7fca31d808e12e65d07b" Jul 31 23:31:55.934916 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.934836 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3ce7644f5b2a047cad03eec86202ee35d4c219307fa7fca31d808e12e65d07b"} err="failed to get container status \"e3ce7644f5b2a047cad03eec86202ee35d4c219307fa7fca31d808e12e65d07b\": rpc error: code = NotFound desc = could not find container \"e3ce7644f5b2a047cad03eec86202ee35d4c219307fa7fca31d808e12e65d07b\": container with ID starting with e3ce7644f5b2a047cad03eec86202ee35d4c219307fa7fca31d808e12e65d07b not found: ID does not exist" Jul 31 23:31:55.934916 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.934862 2561 scope.go:117] "RemoveContainer" containerID="3554dd0fa8f9a2f641a1459df74506ad44ce5a1b7b23dcd121c70a921792b947" Jul 31 23:31:55.935201 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:31:55.935175 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3554dd0fa8f9a2f641a1459df74506ad44ce5a1b7b23dcd121c70a921792b947\": container with ID starting with 3554dd0fa8f9a2f641a1459df74506ad44ce5a1b7b23dcd121c70a921792b947 not found: ID does not exist" containerID="3554dd0fa8f9a2f641a1459df74506ad44ce5a1b7b23dcd121c70a921792b947" Jul 31 23:31:55.935268 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.935213 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3554dd0fa8f9a2f641a1459df74506ad44ce5a1b7b23dcd121c70a921792b947"} err="failed to get container status \"3554dd0fa8f9a2f641a1459df74506ad44ce5a1b7b23dcd121c70a921792b947\": rpc error: code = NotFound desc = could not find container \"3554dd0fa8f9a2f641a1459df74506ad44ce5a1b7b23dcd121c70a921792b947\": container with ID starting with 3554dd0fa8f9a2f641a1459df74506ad44ce5a1b7b23dcd121c70a921792b947 not found: ID does not exist" Jul 31 23:31:55.938365 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.938338 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn"] Jul 31 23:31:55.941926 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:55.941900 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-124aaef2-openshift-default-6b48ccd6c6-2cwgn"] Jul 31 23:31:56.113763 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:56.113722 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:56.133555 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:56.133521 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="434d4206-fc4b-4154-b4a1-de24d84e16ab" path="/var/lib/kubelet/pods/434d4206-fc4b-4154-b4a1-de24d84e16ab/volumes" Jul 31 23:31:56.220950 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:56.220855 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:56.315226 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:56.315192 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:31:56.444484 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:31:56.444446 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:32:02.185662 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:02.185624 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:32:02.290291 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:02.290260 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:32:30.369815 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:30.369780 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm"] Jul 31 23:32:30.370475 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:30.369965 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" podUID="ed49b28e-86a7-4c1f-8e54-1de6f33c54c0" containerName="manager" containerID="cri-o://e3e4bbcd050235b78e1e7ac356d2ec479ee19df046bda8aebd437bcb5df26a3a" gracePeriod=10 Jul 31 23:32:30.626315 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:30.626240 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" Jul 31 23:32:30.742787 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:30.742745 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rb5mb\" (UniqueName: \"kubernetes.io/projected/ed49b28e-86a7-4c1f-8e54-1de6f33c54c0-kube-api-access-rb5mb\") pod \"ed49b28e-86a7-4c1f-8e54-1de6f33c54c0\" (UID: \"ed49b28e-86a7-4c1f-8e54-1de6f33c54c0\") " Jul 31 23:32:30.743001 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:30.742816 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed49b28e-86a7-4c1f-8e54-1de6f33c54c0-extensions-socket-volume\") pod \"ed49b28e-86a7-4c1f-8e54-1de6f33c54c0\" (UID: \"ed49b28e-86a7-4c1f-8e54-1de6f33c54c0\") " Jul 31 23:32:30.743288 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:30.743258 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed49b28e-86a7-4c1f-8e54-1de6f33c54c0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ed49b28e-86a7-4c1f-8e54-1de6f33c54c0" (UID: "ed49b28e-86a7-4c1f-8e54-1de6f33c54c0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:32:30.745066 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:30.745039 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed49b28e-86a7-4c1f-8e54-1de6f33c54c0-kube-api-access-rb5mb" (OuterVolumeSpecName: "kube-api-access-rb5mb") pod "ed49b28e-86a7-4c1f-8e54-1de6f33c54c0" (UID: "ed49b28e-86a7-4c1f-8e54-1de6f33c54c0"). InnerVolumeSpecName "kube-api-access-rb5mb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:32:30.844384 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:30.844339 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rb5mb\" (UniqueName: \"kubernetes.io/projected/ed49b28e-86a7-4c1f-8e54-1de6f33c54c0-kube-api-access-rb5mb\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:32:30.844384 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:30.844376 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed49b28e-86a7-4c1f-8e54-1de6f33c54c0-extensions-socket-volume\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:32:31.102657 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:31.102622 2561 generic.go:356] "Generic (PLEG): container finished" podID="ed49b28e-86a7-4c1f-8e54-1de6f33c54c0" containerID="e3e4bbcd050235b78e1e7ac356d2ec479ee19df046bda8aebd437bcb5df26a3a" exitCode=0 Jul 31 23:32:31.102848 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:31.102668 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" Jul 31 23:32:31.102848 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:31.102701 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" event={"ID":"ed49b28e-86a7-4c1f-8e54-1de6f33c54c0","Type":"ContainerDied","Data":"e3e4bbcd050235b78e1e7ac356d2ec479ee19df046bda8aebd437bcb5df26a3a"} Jul 31 23:32:31.102848 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:31.102757 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm" event={"ID":"ed49b28e-86a7-4c1f-8e54-1de6f33c54c0","Type":"ContainerDied","Data":"a849ced3fd498d1d94916e4b0b7e54f9df5f80cc23b722485c032732cf611584"} Jul 31 23:32:31.102848 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:31.102780 2561 scope.go:117] "RemoveContainer" containerID="e3e4bbcd050235b78e1e7ac356d2ec479ee19df046bda8aebd437bcb5df26a3a" Jul 31 23:32:31.113391 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:31.113369 2561 scope.go:117] "RemoveContainer" containerID="e3e4bbcd050235b78e1e7ac356d2ec479ee19df046bda8aebd437bcb5df26a3a" Jul 31 23:32:31.113742 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:32:31.113721 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3e4bbcd050235b78e1e7ac356d2ec479ee19df046bda8aebd437bcb5df26a3a\": container with ID starting with e3e4bbcd050235b78e1e7ac356d2ec479ee19df046bda8aebd437bcb5df26a3a not found: ID does not exist" containerID="e3e4bbcd050235b78e1e7ac356d2ec479ee19df046bda8aebd437bcb5df26a3a" Jul 31 23:32:31.113825 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:31.113751 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3e4bbcd050235b78e1e7ac356d2ec479ee19df046bda8aebd437bcb5df26a3a"} err="failed to get container status \"e3e4bbcd050235b78e1e7ac356d2ec479ee19df046bda8aebd437bcb5df26a3a\": rpc error: code = NotFound desc = could not find container \"e3e4bbcd050235b78e1e7ac356d2ec479ee19df046bda8aebd437bcb5df26a3a\": container with ID starting with e3e4bbcd050235b78e1e7ac356d2ec479ee19df046bda8aebd437bcb5df26a3a not found: ID does not exist" Jul 31 23:32:31.124444 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:31.124409 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm"] Jul 31 23:32:31.128081 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:31.128050 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-5wxlm"] Jul 31 23:32:32.132911 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:32:32.132876 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed49b28e-86a7-4c1f-8e54-1de6f33c54c0" path="/var/lib/kubelet/pods/ed49b28e-86a7-4c1f-8e54-1de6f33c54c0/volumes" Jul 31 23:33:45.953256 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:45.953218 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v"] Jul 31 23:33:45.953764 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:45.953669 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerName="istio-proxy" Jul 31 23:33:45.953764 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:45.953681 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerName="istio-proxy" Jul 31 23:33:45.953764 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:45.953690 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerName="istio-proxy" Jul 31 23:33:45.953764 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:45.953696 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerName="istio-proxy" Jul 31 23:33:45.953764 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:45.953704 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed49b28e-86a7-4c1f-8e54-1de6f33c54c0" containerName="manager" Jul 31 23:33:45.953764 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:45.953710 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed49b28e-86a7-4c1f-8e54-1de6f33c54c0" containerName="manager" Jul 31 23:33:45.953764 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:45.953727 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerName="istio-proxy" Jul 31 23:33:45.953764 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:45.953734 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="80ecf28a-7902-4201-aa05-9b727721bcbe" containerName="istio-proxy" Jul 31 23:33:45.954019 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:45.953793 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerName="istio-proxy" Jul 31 23:33:45.954019 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:45.953801 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed49b28e-86a7-4c1f-8e54-1de6f33c54c0" containerName="manager" Jul 31 23:33:45.957068 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:45.957049 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v" Jul 31 23:33:45.960223 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:45.960193 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mcgdw\"" Jul 31 23:33:45.968758 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:45.968722 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v"] Jul 31 23:33:46.112000 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:46.111954 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk67k\" (UniqueName: \"kubernetes.io/projected/ca14563e-f9db-4f8b-832c-73cca13670ae-kube-api-access-hk67k\") pod \"kuadrant-operator-controller-manager-68cf589589-96b8v\" (UID: \"ca14563e-f9db-4f8b-832c-73cca13670ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v" Jul 31 23:33:46.112204 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:46.112052 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca14563e-f9db-4f8b-832c-73cca13670ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-96b8v\" (UID: \"ca14563e-f9db-4f8b-832c-73cca13670ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v" Jul 31 23:33:46.213422 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:46.213322 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca14563e-f9db-4f8b-832c-73cca13670ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-96b8v\" (UID: \"ca14563e-f9db-4f8b-832c-73cca13670ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v" Jul 31 23:33:46.213612 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:46.213440 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk67k\" (UniqueName: \"kubernetes.io/projected/ca14563e-f9db-4f8b-832c-73cca13670ae-kube-api-access-hk67k\") pod \"kuadrant-operator-controller-manager-68cf589589-96b8v\" (UID: \"ca14563e-f9db-4f8b-832c-73cca13670ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v" Jul 31 23:33:46.213802 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:46.213779 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca14563e-f9db-4f8b-832c-73cca13670ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-96b8v\" (UID: \"ca14563e-f9db-4f8b-832c-73cca13670ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v" Jul 31 23:33:46.221694 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:46.221664 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk67k\" (UniqueName: \"kubernetes.io/projected/ca14563e-f9db-4f8b-832c-73cca13670ae-kube-api-access-hk67k\") pod \"kuadrant-operator-controller-manager-68cf589589-96b8v\" (UID: \"ca14563e-f9db-4f8b-832c-73cca13670ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v" Jul 31 23:33:46.270218 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:46.270164 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v" Jul 31 23:33:46.418029 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:46.418006 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v"] Jul 31 23:33:46.420695 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:33:46.420665 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca14563e_f9db_4f8b_832c_73cca13670ae.slice/crio-1b7f2b3ae52664a26c84187f7dda82c0a015e12acfffb45added0f9c3ff6548c WatchSource:0}: Error finding container 1b7f2b3ae52664a26c84187f7dda82c0a015e12acfffb45added0f9c3ff6548c: Status 404 returned error can't find the container with id 1b7f2b3ae52664a26c84187f7dda82c0a015e12acfffb45added0f9c3ff6548c Jul 31 23:33:46.500300 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:46.500267 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v" event={"ID":"ca14563e-f9db-4f8b-832c-73cca13670ae","Type":"ContainerStarted","Data":"63c29296a8a8b0d7a3a7f060253262d675a41b9e1bba8f24a90fee8aead37717"} Jul 31 23:33:46.500480 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:46.500321 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v" event={"ID":"ca14563e-f9db-4f8b-832c-73cca13670ae","Type":"ContainerStarted","Data":"1b7f2b3ae52664a26c84187f7dda82c0a015e12acfffb45added0f9c3ff6548c"} Jul 31 23:33:46.500480 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:46.500420 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v" Jul 31 23:33:46.525629 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:46.525410 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v" podStartSLOduration=1.525381897 podStartE2EDuration="1.525381897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:33:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:33:46.521679452 +0000 UTC m=+2182.994123635" watchObservedRunningTime="2026-07-31 23:33:46.525381897 +0000 UTC m=+2182.997826084" Jul 31 23:33:57.508257 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:57.508221 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-96b8v" Jul 31 23:33:58.945293 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:58.945258 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:33:59.268382 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:59.268284 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:33:59.658351 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:33:59.658315 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:34:03.939781 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:03.939727 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:34:04.211342 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.211251 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc"] Jul 31 23:34:04.212043 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.212025 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="434d4206-fc4b-4154-b4a1-de24d84e16ab" containerName="istio-proxy" Jul 31 23:34:04.215376 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.215358 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.217937 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.217917 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0a726cba-openshift-default-dockercfg-7w82n\"" Jul 31 23:34:04.228673 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.228647 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc"] Jul 31 23:34:04.232196 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.232166 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:34:04.300564 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.300517 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsblv\" (UniqueName: \"kubernetes.io/projected/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-kube-api-access-xsblv\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.300800 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.300636 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-podinfo\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.300800 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.300691 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-data\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.300800 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.300746 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-workload-certs\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.301028 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.300806 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istiod-ca-cert\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.301028 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.300872 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-credential-socket\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.301028 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.300895 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-workload-socket\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.301028 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.300996 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-envoy\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.301170 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.301042 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-token\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.400374 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.400334 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:34:04.402631 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.402602 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-credential-socket\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.402809 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.402645 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-workload-socket\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.402809 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.402713 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-envoy\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.402809 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.402742 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-token\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.402809 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.402796 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsblv\" (UniqueName: \"kubernetes.io/projected/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-kube-api-access-xsblv\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.403031 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.402828 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-podinfo\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.403031 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.402862 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-data\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.403031 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.402904 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-workload-certs\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.403031 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.402966 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istiod-ca-cert\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.403031 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.403013 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-credential-socket\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.403287 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.403062 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-workload-socket\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.403287 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.403273 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-data\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.403404 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.403339 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-workload-certs\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.403720 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.403698 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istiod-ca-cert\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.405294 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.405273 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-envoy\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.405404 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.405384 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-podinfo\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.410424 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.410396 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-token\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.410671 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.410650 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsblv\" (UniqueName: \"kubernetes.io/projected/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-kube-api-access-xsblv\") pod \"e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.527179 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.527077 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:04.680189 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.680140 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc"] Jul 31 23:34:04.682563 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:34:04.682529 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6abbedb_dad1_44c3_b81e_1cf5a46e2dcf.slice/crio-f0ff9740222afe5182cef133dd234d80ab9930a4829b83600cf0ac9be6467799 WatchSource:0}: Error finding container f0ff9740222afe5182cef133dd234d80ab9930a4829b83600cf0ac9be6467799: Status 404 returned error can't find the container with id f0ff9740222afe5182cef133dd234d80ab9930a4829b83600cf0ac9be6467799 Jul 31 23:34:04.685114 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.685074 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:34:04.685210 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.685173 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:34:04.685251 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:04.685225 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:34:05.605453 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:05.605416 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" event={"ID":"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf","Type":"ContainerStarted","Data":"438b19f09cdc6afb9a3fa9f92572fa744d415a93a6c85f45c5db12fb270b8759"} Jul 31 23:34:05.605921 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:05.605485 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" event={"ID":"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf","Type":"ContainerStarted","Data":"f0ff9740222afe5182cef133dd234d80ab9930a4829b83600cf0ac9be6467799"} Jul 31 23:34:05.626806 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:05.626750 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" podStartSLOduration=1.626731774 podStartE2EDuration="1.626731774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:34:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:34:05.624406968 +0000 UTC m=+2202.096851187" watchObservedRunningTime="2026-07-31 23:34:05.626731774 +0000 UTC m=+2202.099176038" Jul 31 23:34:06.527618 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:06.527582 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:06.533116 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:06.533088 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:06.610985 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:06.610943 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:06.612047 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:06.612021 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:07.508111 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:07.508077 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:34:07.597940 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:07.597899 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:34:08.703518 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:08.703464 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:34:09.032342 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:09.032243 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:34:09.278870 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:09.278832 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:34:09.381028 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:09.380981 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:34:21.612642 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:21.612604 2561 patch_prober.go:28] interesting pod/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 31 23:34:21.613117 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:21.612680 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" podUID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 31 23:34:21.700112 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:21.700028 2561 generic.go:356] "Generic (PLEG): container finished" podID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerID="438b19f09cdc6afb9a3fa9f92572fa744d415a93a6c85f45c5db12fb270b8759" exitCode=0 Jul 31 23:34:21.700112 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:21.700103 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" event={"ID":"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf","Type":"ContainerDied","Data":"438b19f09cdc6afb9a3fa9f92572fa744d415a93a6c85f45c5db12fb270b8759"} Jul 31 23:34:21.700675 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:21.700656 2561 scope.go:117] "RemoveContainer" containerID="438b19f09cdc6afb9a3fa9f92572fa744d415a93a6c85f45c5db12fb270b8759" Jul 31 23:34:21.701871 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:21.701842 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:34:21.701944 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:21.701917 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:34:21.701986 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:21.701951 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:34:22.708156 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:22.708119 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" event={"ID":"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf","Type":"ContainerStarted","Data":"1df64cbf8a41d18e867d41e34a564d165d18c87dde1aac023ac10c7f48a4a8f4"} Jul 31 23:34:23.527618 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:23.527574 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:23.527946 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:23.527921 2561 patch_prober.go:28] interesting pod/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 31 23:34:23.528009 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:23.527986 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" podUID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 31 23:34:24.527965 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:24.527923 2561 patch_prober.go:28] interesting pod/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 31 23:34:24.528476 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:24.527991 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" podUID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 31 23:34:25.528566 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:25.528526 2561 patch_prober.go:28] interesting pod/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 31 23:34:25.528988 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:25.528584 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" podUID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 31 23:34:26.528150 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:26.528106 2561 patch_prober.go:28] interesting pod/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 31 23:34:26.528384 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:26.528170 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" podUID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 31 23:34:27.528287 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:27.528243 2561 patch_prober.go:28] interesting pod/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 31 23:34:27.528833 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:27.528310 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" podUID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 31 23:34:28.532570 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:28.532531 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:28.533065 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:28.532937 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:28.533628 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:28.533606 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:34:37.569238 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:37.569201 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:34:50.394448 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.394361 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9"] Jul 31 23:34:50.398086 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.398067 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.401701 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.401673 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-0a726cba\"/\"openshift-service-ca.crt\"" Jul 31 23:34:50.401701 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.401691 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-0a726cba\"/\"kube-root-ca.crt\"" Jul 31 23:34:50.401701 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.401697 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-0a726cba\"/\"default-dockercfg-djfdb\"" Jul 31 23:34:50.401925 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.401693 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-0a726cba\"/\"e2e-lifecycle-model-0a726cba-kserve-self-signed-certs\"" Jul 31 23:34:50.407969 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.407947 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9"] Jul 31 23:34:50.489624 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.489587 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:34:50.500797 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.500767 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c77f1838-08e6-4a08-bb37-baa45e2de52e-tls-certs\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.500922 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.500813 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-model-cache\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.500922 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.500872 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-home\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.500998 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.500928 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-tmp-dir\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.501078 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.501060 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-dshm\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.501115 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.501087 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbqll\" (UniqueName: \"kubernetes.io/projected/c77f1838-08e6-4a08-bb37-baa45e2de52e-kube-api-access-zbqll\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.602025 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.601976 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-dshm\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.602025 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.602024 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbqll\" (UniqueName: \"kubernetes.io/projected/c77f1838-08e6-4a08-bb37-baa45e2de52e-kube-api-access-zbqll\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.602257 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.602066 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c77f1838-08e6-4a08-bb37-baa45e2de52e-tls-certs\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.602257 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.602091 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-model-cache\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.602257 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.602114 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-home\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.602257 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.602153 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-tmp-dir\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.602583 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.602562 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-model-cache\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.602678 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.602580 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-home\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.602678 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.602664 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-tmp-dir\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.604364 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.604341 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-dshm\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.604654 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.604633 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c77f1838-08e6-4a08-bb37-baa45e2de52e-tls-certs\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.609776 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.609751 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbqll\" (UniqueName: \"kubernetes.io/projected/c77f1838-08e6-4a08-bb37-baa45e2de52e-kube-api-access-zbqll\") pod \"e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.709537 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.709397 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:34:50.861012 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:50.860983 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9"] Jul 31 23:34:50.862724 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:34:50.862694 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc77f1838_08e6_4a08_bb37_baa45e2de52e.slice/crio-afac0e5225224209d25adcee07619796db2dde6708a6161d2b16585a5ff32942 WatchSource:0}: Error finding container afac0e5225224209d25adcee07619796db2dde6708a6161d2b16585a5ff32942: Status 404 returned error can't find the container with id afac0e5225224209d25adcee07619796db2dde6708a6161d2b16585a5ff32942 Jul 31 23:34:51.869536 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:51.869471 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" event={"ID":"c77f1838-08e6-4a08-bb37-baa45e2de52e","Type":"ContainerStarted","Data":"c7172571ee71c7b14b2c0df30f4f30d35084dc63d878e686fd9b67da0432c481"} Jul 31 23:34:51.869536 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:51.869532 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" event={"ID":"c77f1838-08e6-4a08-bb37-baa45e2de52e","Type":"ContainerStarted","Data":"afac0e5225224209d25adcee07619796db2dde6708a6161d2b16585a5ff32942"} Jul 31 23:34:51.885315 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:34:51.885257 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" podStartSLOduration=1.8852386719999998 podStartE2EDuration="1.885238672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:34:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:34:51.884339536 +0000 UTC m=+2248.356783749" watchObservedRunningTime="2026-07-31 23:34:51.885238672 +0000 UTC m=+2248.357682855" Jul 31 23:35:00.709773 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:00.709619 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:35:00.710327 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:00.709809 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:35:00.722970 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:00.722935 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:35:00.930645 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:00.930607 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:35:32.390691 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:32.390656 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:35:33.747673 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:33.747634 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:35:35.535152 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:35.535117 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:35:40.519620 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:40.519582 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9"] Jul 31 23:35:40.566033 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:40.565993 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:35:55.539695 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.539636 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" podUID="c77f1838-08e6-4a08-bb37-baa45e2de52e" containerName="main" containerID="cri-o://c7172571ee71c7b14b2c0df30f4f30d35084dc63d878e686fd9b67da0432c481" gracePeriod=45 Jul 31 23:35:55.800286 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.800219 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:35:55.896569 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.896491 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbqll\" (UniqueName: \"kubernetes.io/projected/c77f1838-08e6-4a08-bb37-baa45e2de52e-kube-api-access-zbqll\") pod \"c77f1838-08e6-4a08-bb37-baa45e2de52e\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " Jul 31 23:35:55.896755 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.896702 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-dshm\") pod \"c77f1838-08e6-4a08-bb37-baa45e2de52e\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " Jul 31 23:35:55.896755 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.896726 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-tmp-dir\") pod \"c77f1838-08e6-4a08-bb37-baa45e2de52e\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " Jul 31 23:35:55.896755 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.896747 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-model-cache\") pod \"c77f1838-08e6-4a08-bb37-baa45e2de52e\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " Jul 31 23:35:55.896919 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.896782 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c77f1838-08e6-4a08-bb37-baa45e2de52e-tls-certs\") pod \"c77f1838-08e6-4a08-bb37-baa45e2de52e\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " Jul 31 23:35:55.896919 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.896801 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-home\") pod \"c77f1838-08e6-4a08-bb37-baa45e2de52e\" (UID: \"c77f1838-08e6-4a08-bb37-baa45e2de52e\") " Jul 31 23:35:55.897076 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.897036 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-model-cache" (OuterVolumeSpecName: "model-cache") pod "c77f1838-08e6-4a08-bb37-baa45e2de52e" (UID: "c77f1838-08e6-4a08-bb37-baa45e2de52e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:35:55.897143 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.897047 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c77f1838-08e6-4a08-bb37-baa45e2de52e" (UID: "c77f1838-08e6-4a08-bb37-baa45e2de52e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:35:55.897143 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.897113 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-home" (OuterVolumeSpecName: "home") pod "c77f1838-08e6-4a08-bb37-baa45e2de52e" (UID: "c77f1838-08e6-4a08-bb37-baa45e2de52e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:35:55.897297 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.897279 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-home\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:35:55.897357 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.897299 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-tmp-dir\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:35:55.897357 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.897311 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-model-cache\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:35:55.898953 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.898922 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-dshm" (OuterVolumeSpecName: "dshm") pod "c77f1838-08e6-4a08-bb37-baa45e2de52e" (UID: "c77f1838-08e6-4a08-bb37-baa45e2de52e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:35:55.898953 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.898950 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c77f1838-08e6-4a08-bb37-baa45e2de52e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c77f1838-08e6-4a08-bb37-baa45e2de52e" (UID: "c77f1838-08e6-4a08-bb37-baa45e2de52e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 23:35:55.899107 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.898953 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c77f1838-08e6-4a08-bb37-baa45e2de52e-kube-api-access-zbqll" (OuterVolumeSpecName: "kube-api-access-zbqll") pod "c77f1838-08e6-4a08-bb37-baa45e2de52e" (UID: "c77f1838-08e6-4a08-bb37-baa45e2de52e"). InnerVolumeSpecName "kube-api-access-zbqll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:35:55.998672 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.998629 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c77f1838-08e6-4a08-bb37-baa45e2de52e-dshm\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:35:55.998672 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.998666 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c77f1838-08e6-4a08-bb37-baa45e2de52e-tls-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:35:55.998672 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:55.998677 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbqll\" (UniqueName: \"kubernetes.io/projected/c77f1838-08e6-4a08-bb37-baa45e2de52e-kube-api-access-zbqll\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:35:56.225978 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:56.225943 2561 generic.go:356] "Generic (PLEG): container finished" podID="c77f1838-08e6-4a08-bb37-baa45e2de52e" containerID="c7172571ee71c7b14b2c0df30f4f30d35084dc63d878e686fd9b67da0432c481" exitCode=0 Jul 31 23:35:56.226187 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:56.225984 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" Jul 31 23:35:56.226187 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:56.226024 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" event={"ID":"c77f1838-08e6-4a08-bb37-baa45e2de52e","Type":"ContainerDied","Data":"c7172571ee71c7b14b2c0df30f4f30d35084dc63d878e686fd9b67da0432c481"} Jul 31 23:35:56.226187 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:56.226065 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9" event={"ID":"c77f1838-08e6-4a08-bb37-baa45e2de52e","Type":"ContainerDied","Data":"afac0e5225224209d25adcee07619796db2dde6708a6161d2b16585a5ff32942"} Jul 31 23:35:56.226187 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:56.226083 2561 scope.go:117] "RemoveContainer" containerID="c7172571ee71c7b14b2c0df30f4f30d35084dc63d878e686fd9b67da0432c481" Jul 31 23:35:56.235350 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:56.235331 2561 scope.go:117] "RemoveContainer" containerID="c7172571ee71c7b14b2c0df30f4f30d35084dc63d878e686fd9b67da0432c481" Jul 31 23:35:56.235687 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:35:56.235665 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7172571ee71c7b14b2c0df30f4f30d35084dc63d878e686fd9b67da0432c481\": container with ID starting with c7172571ee71c7b14b2c0df30f4f30d35084dc63d878e686fd9b67da0432c481 not found: ID does not exist" containerID="c7172571ee71c7b14b2c0df30f4f30d35084dc63d878e686fd9b67da0432c481" Jul 31 23:35:56.235777 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:56.235699 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7172571ee71c7b14b2c0df30f4f30d35084dc63d878e686fd9b67da0432c481"} err="failed to get container status \"c7172571ee71c7b14b2c0df30f4f30d35084dc63d878e686fd9b67da0432c481\": rpc error: code = NotFound desc = could not find container \"c7172571ee71c7b14b2c0df30f4f30d35084dc63d878e686fd9b67da0432c481\": container with ID starting with c7172571ee71c7b14b2c0df30f4f30d35084dc63d878e686fd9b67da0432c481 not found: ID does not exist" Jul 31 23:35:56.242357 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:56.242330 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9"] Jul 31 23:35:56.244322 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:56.244299 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-0a726cba/e2e-lifecycle-model-0a726cba-kserve-5c6c6c9467-qjzw9"] Jul 31 23:35:58.134225 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:35:58.134190 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c77f1838-08e6-4a08-bb37-baa45e2de52e" path="/var/lib/kubelet/pods/c77f1838-08e6-4a08-bb37-baa45e2de52e/volumes" Jul 31 23:36:16.657655 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:16.657570 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:17.725902 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:17.725860 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:23.122101 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:23.122062 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:23.175554 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:23.175489 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:23.366067 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:23.366034 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc"] Jul 31 23:36:23.366294 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:23.366251 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" podUID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerName="istio-proxy" containerID="cri-o://1df64cbf8a41d18e867d41e34a564d165d18c87dde1aac023ac10c7f48a4a8f4" gracePeriod=30 Jul 31 23:36:23.381424 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:23.381351 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:23.614262 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:23.614232 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:28.534319 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.534280 2561 patch_prober.go:28] interesting pod/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 31 23:36:28.534761 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.534343 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" podUID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 31 23:36:28.623318 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.623285 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:36:28.721660 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.721568 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-workload-socket\") pod \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " Jul 31 23:36:28.721660 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.721620 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-workload-certs\") pod \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " Jul 31 23:36:28.721881 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.721683 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-credential-socket\") pod \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " Jul 31 23:36:28.721881 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.721722 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istiod-ca-cert\") pod \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " Jul 31 23:36:28.721881 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.721769 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-token\") pod \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " Jul 31 23:36:28.721881 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.721819 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-podinfo\") pod \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " Jul 31 23:36:28.721881 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.721844 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-data\") pod \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " Jul 31 23:36:28.722133 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.721906 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsblv\" (UniqueName: \"kubernetes.io/projected/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-kube-api-access-xsblv\") pod \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " Jul 31 23:36:28.722133 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.721929 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-envoy\") pod \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\" (UID: \"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf\") " Jul 31 23:36:28.722133 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.721897 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" (UID: "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:36:28.722133 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.721933 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" (UID: "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:36:28.722133 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.721992 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" (UID: "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:36:28.722133 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.722100 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-data" (OuterVolumeSpecName: "istio-data") pod "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" (UID: "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:36:28.722450 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.722263 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" (UID: "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:36:28.722450 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.722329 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-data\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:36:28.722450 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.722347 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-workload-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:36:28.722450 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.722362 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-workload-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:36:28.722450 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.722375 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-credential-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:36:28.724238 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.724206 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-token" (OuterVolumeSpecName: "istio-token") pod "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" (UID: "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:36:28.724368 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.724266 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-kube-api-access-xsblv" (OuterVolumeSpecName: "kube-api-access-xsblv") pod "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" (UID: "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf"). InnerVolumeSpecName "kube-api-access-xsblv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:36:28.724368 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.724299 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" (UID: "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 23:36:28.724368 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.724343 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" (UID: "f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:36:28.823297 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.823255 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istiod-ca-cert\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:36:28.823297 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.823294 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-token\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:36:28.823550 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.823309 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-podinfo\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:36:28.823550 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.823321 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsblv\" (UniqueName: \"kubernetes.io/projected/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-kube-api-access-xsblv\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:36:28.823550 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:28.823336 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf-istio-envoy\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:36:29.403729 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:29.403690 2561 generic.go:356] "Generic (PLEG): container finished" podID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerID="1df64cbf8a41d18e867d41e34a564d165d18c87dde1aac023ac10c7f48a4a8f4" exitCode=0 Jul 31 23:36:29.403933 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:29.403736 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" Jul 31 23:36:29.403933 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:29.403777 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" event={"ID":"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf","Type":"ContainerDied","Data":"1df64cbf8a41d18e867d41e34a564d165d18c87dde1aac023ac10c7f48a4a8f4"} Jul 31 23:36:29.403933 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:29.403836 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc" event={"ID":"f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf","Type":"ContainerDied","Data":"f0ff9740222afe5182cef133dd234d80ab9930a4829b83600cf0ac9be6467799"} Jul 31 23:36:29.403933 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:29.403862 2561 scope.go:117] "RemoveContainer" containerID="1df64cbf8a41d18e867d41e34a564d165d18c87dde1aac023ac10c7f48a4a8f4" Jul 31 23:36:29.413850 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:29.413831 2561 scope.go:117] "RemoveContainer" containerID="438b19f09cdc6afb9a3fa9f92572fa744d415a93a6c85f45c5db12fb270b8759" Jul 31 23:36:29.422878 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:29.422843 2561 scope.go:117] "RemoveContainer" containerID="1df64cbf8a41d18e867d41e34a564d165d18c87dde1aac023ac10c7f48a4a8f4" Jul 31 23:36:29.423587 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:36:29.423390 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1df64cbf8a41d18e867d41e34a564d165d18c87dde1aac023ac10c7f48a4a8f4\": container with ID starting with 1df64cbf8a41d18e867d41e34a564d165d18c87dde1aac023ac10c7f48a4a8f4 not found: ID does not exist" containerID="1df64cbf8a41d18e867d41e34a564d165d18c87dde1aac023ac10c7f48a4a8f4" Jul 31 23:36:29.423587 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:29.423447 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1df64cbf8a41d18e867d41e34a564d165d18c87dde1aac023ac10c7f48a4a8f4"} err="failed to get container status \"1df64cbf8a41d18e867d41e34a564d165d18c87dde1aac023ac10c7f48a4a8f4\": rpc error: code = NotFound desc = could not find container \"1df64cbf8a41d18e867d41e34a564d165d18c87dde1aac023ac10c7f48a4a8f4\": container with ID starting with 1df64cbf8a41d18e867d41e34a564d165d18c87dde1aac023ac10c7f48a4a8f4 not found: ID does not exist" Jul 31 23:36:29.423587 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:29.423476 2561 scope.go:117] "RemoveContainer" containerID="438b19f09cdc6afb9a3fa9f92572fa744d415a93a6c85f45c5db12fb270b8759" Jul 31 23:36:29.423969 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:36:29.423938 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"438b19f09cdc6afb9a3fa9f92572fa744d415a93a6c85f45c5db12fb270b8759\": container with ID starting with 438b19f09cdc6afb9a3fa9f92572fa744d415a93a6c85f45c5db12fb270b8759 not found: ID does not exist" containerID="438b19f09cdc6afb9a3fa9f92572fa744d415a93a6c85f45c5db12fb270b8759" Jul 31 23:36:29.424096 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:29.423995 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"438b19f09cdc6afb9a3fa9f92572fa744d415a93a6c85f45c5db12fb270b8759"} err="failed to get container status \"438b19f09cdc6afb9a3fa9f92572fa744d415a93a6c85f45c5db12fb270b8759\": rpc error: code = NotFound desc = could not find container \"438b19f09cdc6afb9a3fa9f92572fa744d415a93a6c85f45c5db12fb270b8759\": container with ID starting with 438b19f09cdc6afb9a3fa9f92572fa744d415a93a6c85f45c5db12fb270b8759 not found: ID does not exist" Jul 31 23:36:29.426085 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:29.426062 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc"] Jul 31 23:36:29.429723 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:29.429695 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0a726cba-openshift-default-bfc7c7444-2jzsc"] Jul 31 23:36:30.132616 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:30.132582 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" path="/var/lib/kubelet/pods/f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf/volumes" Jul 31 23:36:34.858698 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:34.858662 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:34.957291 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:34.957253 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:35.057373 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:35.057328 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:35.639635 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:35.639598 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:35.739351 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:35.739312 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:35.839697 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:35.839658 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:37.023764 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:37.023728 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:43.104358 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:43.104315 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:43.226789 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:43.226756 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:43.764399 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:43.764366 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:50.037712 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:50.037670 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:50.163778 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:50.163743 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:36:50.294618 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:36:50.294494 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:00.742862 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:00.742817 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:00.963021 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:00.962976 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:21.884765 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:21.884724 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:27.747608 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:27.747569 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:27.808193 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:27.808161 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:28.088160 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.088116 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:28.373782 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.373691 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h"] Jul 31 23:37:28.374153 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.374139 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerName="istio-proxy" Jul 31 23:37:28.374196 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.374155 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerName="istio-proxy" Jul 31 23:37:28.374196 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.374171 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c77f1838-08e6-4a08-bb37-baa45e2de52e" containerName="main" Jul 31 23:37:28.374196 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.374179 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c77f1838-08e6-4a08-bb37-baa45e2de52e" containerName="main" Jul 31 23:37:28.374292 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.374201 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerName="istio-proxy" Jul 31 23:37:28.374292 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.374206 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerName="istio-proxy" Jul 31 23:37:28.374292 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.374275 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerName="istio-proxy" Jul 31 23:37:28.374292 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.374285 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6abbedb-dad1-44c3-b81e-1cf5a46e2dcf" containerName="istio-proxy" Jul 31 23:37:28.374415 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.374294 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c77f1838-08e6-4a08-bb37-baa45e2de52e" containerName="main" Jul 31 23:37:28.380820 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.380778 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.384292 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.384263 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-02002b-openshift-default-dockercfg-gj8pq\"" Jul 31 23:37:28.392061 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.392026 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h"] Jul 31 23:37:28.392577 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.392547 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:28.431226 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.431185 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-istio-data\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.431226 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.431224 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95c885a7-17fb-4483-a517-33f3f886ac23-istio-podinfo\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.431449 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.431250 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-workload-certs\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.431449 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.431320 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-workload-socket\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.431449 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.431365 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95c885a7-17fb-4483-a517-33f3f886ac23-istio-token\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.431449 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.431382 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-credential-socket\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.431449 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.431410 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-istio-envoy\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.431659 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.431482 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9lwr\" (UniqueName: \"kubernetes.io/projected/95c885a7-17fb-4483-a517-33f3f886ac23-kube-api-access-r9lwr\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.431659 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.431566 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95c885a7-17fb-4483-a517-33f3f886ac23-istiod-ca-cert\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.532620 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.532571 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-workload-socket\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.532784 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.532634 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95c885a7-17fb-4483-a517-33f3f886ac23-istio-token\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.532784 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.532664 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-credential-socket\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.532784 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.532686 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-istio-envoy\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.532971 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.532809 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9lwr\" (UniqueName: \"kubernetes.io/projected/95c885a7-17fb-4483-a517-33f3f886ac23-kube-api-access-r9lwr\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.532971 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.532861 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95c885a7-17fb-4483-a517-33f3f886ac23-istiod-ca-cert\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.532971 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.532943 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-istio-data\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.532971 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.532969 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95c885a7-17fb-4483-a517-33f3f886ac23-istio-podinfo\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.533233 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.533002 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-workload-certs\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.533233 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.533087 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-credential-socket\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.533233 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.533102 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-workload-socket\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.533335 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.533274 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-istio-data\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.533378 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.533345 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-workload-certs\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.533646 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.533627 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95c885a7-17fb-4483-a517-33f3f886ac23-istiod-ca-cert\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.535113 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.535089 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-istio-envoy\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.535379 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.535360 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95c885a7-17fb-4483-a517-33f3f886ac23-istio-podinfo\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.540845 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.540815 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95c885a7-17fb-4483-a517-33f3f886ac23-istio-token\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.540961 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.540928 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9lwr\" (UniqueName: \"kubernetes.io/projected/95c885a7-17fb-4483-a517-33f3f886ac23-kube-api-access-r9lwr\") pod \"e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.627033 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.626949 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:28.695679 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.695644 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:28.838983 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.838958 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h"] Jul 31 23:37:28.841386 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:37:28.841355 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95c885a7_17fb_4483_a517_33f3f886ac23.slice/crio-bb6793d11f5609c315047bda9977162c409ddbca55bb6d8b4fb59a660478277c WatchSource:0}: Error finding container bb6793d11f5609c315047bda9977162c409ddbca55bb6d8b4fb59a660478277c: Status 404 returned error can't find the container with id bb6793d11f5609c315047bda9977162c409ddbca55bb6d8b4fb59a660478277c Jul 31 23:37:28.843651 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.843617 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:37:28.843743 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.843687 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:37:28.843743 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:28.843718 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:37:29.714419 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:29.714374 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" event={"ID":"95c885a7-17fb-4483-a517-33f3f886ac23","Type":"ContainerStarted","Data":"45ebf0482266987e70b63fe562efef5dda68c50914f5669a186f6c05f28682f9"} Jul 31 23:37:29.714603 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:29.714442 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" event={"ID":"95c885a7-17fb-4483-a517-33f3f886ac23","Type":"ContainerStarted","Data":"bb6793d11f5609c315047bda9977162c409ddbca55bb6d8b4fb59a660478277c"} Jul 31 23:37:29.734438 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:29.734387 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" podStartSLOduration=1.7343705699999998 podStartE2EDuration="1.73437057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:37:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:37:29.732269422 +0000 UTC m=+2406.204713631" watchObservedRunningTime="2026-07-31 23:37:29.73437057 +0000 UTC m=+2406.206814752" Jul 31 23:37:30.696600 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:30.696557 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:30.701325 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:30.701291 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:30.720478 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:30.720440 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:30.721374 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:30.721349 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:31.596090 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:31.596042 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:31.696514 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:31.696442 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:32.719486 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:32.719450 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:32.965984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:32.965944 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:33.163134 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:33.163096 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:33.265403 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:33.265364 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:46.278703 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:46.277788 2561 patch_prober.go:28] interesting pod/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": read tcp 10.134.0.2:46816->10.134.0.52:15021: read: connection reset by peer" start-of-body= Jul 31 23:37:46.278703 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:46.277891 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" podUID="95c885a7-17fb-4483-a517-33f3f886ac23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": read tcp 10.134.0.2:46816->10.134.0.52:15021: read: connection reset by peer" Jul 31 23:37:46.812382 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:46.812349 2561 generic.go:356] "Generic (PLEG): container finished" podID="95c885a7-17fb-4483-a517-33f3f886ac23" containerID="45ebf0482266987e70b63fe562efef5dda68c50914f5669a186f6c05f28682f9" exitCode=0 Jul 31 23:37:46.812586 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:46.812422 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" event={"ID":"95c885a7-17fb-4483-a517-33f3f886ac23","Type":"ContainerDied","Data":"45ebf0482266987e70b63fe562efef5dda68c50914f5669a186f6c05f28682f9"} Jul 31 23:37:46.812917 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:46.812903 2561 scope.go:117] "RemoveContainer" containerID="45ebf0482266987e70b63fe562efef5dda68c50914f5669a186f6c05f28682f9" Jul 31 23:37:46.814073 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:46.814041 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:37:46.814131 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:46.814112 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:37:46.814168 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:46.814145 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:37:47.825775 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:47.825729 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" event={"ID":"95c885a7-17fb-4483-a517-33f3f886ac23","Type":"ContainerStarted","Data":"0d9f61afc8526aa20799d7aa41cff426737dc04e0a8718e05ee1fa2a23453608"} Jul 31 23:37:48.696074 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:48.696039 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:48.696272 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:48.696249 2561 patch_prober.go:28] interesting pod/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 31 23:37:48.696323 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:48.696302 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" podUID="95c885a7-17fb-4483-a517-33f3f886ac23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 31 23:37:49.696810 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:49.696768 2561 patch_prober.go:28] interesting pod/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 31 23:37:49.697302 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:49.696837 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" podUID="95c885a7-17fb-4483-a517-33f3f886ac23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 31 23:37:50.696474 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:50.696433 2561 patch_prober.go:28] interesting pod/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 31 23:37:50.696683 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:50.696494 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" podUID="95c885a7-17fb-4483-a517-33f3f886ac23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 31 23:37:51.696840 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:51.696799 2561 patch_prober.go:28] interesting pod/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 31 23:37:51.697257 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:51.696861 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" podUID="95c885a7-17fb-4483-a517-33f3f886ac23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 31 23:37:52.696371 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:52.696325 2561 patch_prober.go:28] interesting pod/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 31 23:37:52.696621 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:52.696416 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" podUID="95c885a7-17fb-4483-a517-33f3f886ac23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 31 23:37:53.700697 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:53.700658 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:53.701289 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:53.701073 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:53.701712 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:53.701690 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:37:56.558595 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:56.558563 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:56.885939 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:56.885843 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:57.144967 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.144878 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h"] Jul 31 23:37:57.147795 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.147771 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.150560 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.150533 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-3c3302-openshift-default-dockercfg-lj8rk\"" Jul 31 23:37:57.159087 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.159061 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h"] Jul 31 23:37:57.173060 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.173022 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:57.241563 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.241524 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-podinfo\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.241793 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.241616 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-credential-socket\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.241793 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.241655 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-data\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.241793 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.241686 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-envoy\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.241793 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.241715 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-token\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.242038 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.241795 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqxpl\" (UniqueName: \"kubernetes.io/projected/c477bcf3-d54d-4b2a-9c75-79f43a423b01-kube-api-access-bqxpl\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.242038 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.241832 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-workload-certs\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.242038 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.241859 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istiod-ca-cert\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.242038 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.241919 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-workload-socket\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.343522 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.343471 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-credential-socket\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.343739 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.343558 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-data\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.343739 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.343590 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-envoy\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.343739 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.343616 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-token\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.343739 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.343687 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqxpl\" (UniqueName: \"kubernetes.io/projected/c477bcf3-d54d-4b2a-9c75-79f43a423b01-kube-api-access-bqxpl\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.343739 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.343724 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-workload-certs\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.344024 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.343751 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istiod-ca-cert\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.344024 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.343832 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-workload-socket\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.344024 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.343866 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-podinfo\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.344024 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.343974 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-data\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.344414 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.344389 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-workload-socket\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.344623 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.344592 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istiod-ca-cert\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.344761 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.344733 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-workload-certs\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.344829 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.344791 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-credential-socket\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.346443 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.346408 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-envoy\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.346903 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.346878 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-podinfo\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.352046 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.352016 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-token\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.352156 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.352138 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqxpl\" (UniqueName: \"kubernetes.io/projected/c477bcf3-d54d-4b2a-9c75-79f43a423b01-kube-api-access-bqxpl\") pod \"e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.460537 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.460401 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:57.516461 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.516418 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:57.606469 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.606431 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h"] Jul 31 23:37:57.608995 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:37:57.608964 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc477bcf3_d54d_4b2a_9c75_79f43a423b01.slice/crio-6d4e302c305e22e33c7b82b0e1dc52a272799de92b6f88d0db7e8c7cf739436c WatchSource:0}: Error finding container 6d4e302c305e22e33c7b82b0e1dc52a272799de92b6f88d0db7e8c7cf739436c: Status 404 returned error can't find the container with id 6d4e302c305e22e33c7b82b0e1dc52a272799de92b6f88d0db7e8c7cf739436c Jul 31 23:37:57.611347 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.611310 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:37:57.611464 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.611399 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:37:57.611464 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.611455 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:37:57.894167 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.894128 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" event={"ID":"c477bcf3-d54d-4b2a-9c75-79f43a423b01","Type":"ContainerStarted","Data":"b317c413d6eb30bd4fdb295f90ed75fa39e9e2e70fb3f753239cf8f1a8e922ec"} Jul 31 23:37:57.894366 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.894183 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" event={"ID":"c477bcf3-d54d-4b2a-9c75-79f43a423b01","Type":"ContainerStarted","Data":"6d4e302c305e22e33c7b82b0e1dc52a272799de92b6f88d0db7e8c7cf739436c"} Jul 31 23:37:57.916438 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:57.916376 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" podStartSLOduration=0.916357017 podStartE2EDuration="916.357017ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:37:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:37:57.916024209 +0000 UTC m=+2434.388468413" watchObservedRunningTime="2026-07-31 23:37:57.916357017 +0000 UTC m=+2434.388801200" Jul 31 23:37:58.461205 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:58.461164 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:58.466587 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:58.466554 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:58.899734 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:58.899698 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:58.900696 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:58.900672 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:37:59.765306 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:59.765270 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:37:59.865551 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:37:59.865511 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:38:00.899191 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:00.899160 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:38:01.370144 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:01.370113 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:38:01.465220 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:01.465183 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:38:01.536103 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:01.536066 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:38:13.901294 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:13.901207 2561 patch_prober.go:28] interesting pod/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 31 23:38:13.901294 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:13.901265 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" podUID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 31 23:38:14.002174 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:14.002073 2561 generic.go:356] "Generic (PLEG): container finished" podID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" containerID="b317c413d6eb30bd4fdb295f90ed75fa39e9e2e70fb3f753239cf8f1a8e922ec" exitCode=0 Jul 31 23:38:14.002174 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:14.002149 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" event={"ID":"c477bcf3-d54d-4b2a-9c75-79f43a423b01","Type":"ContainerDied","Data":"b317c413d6eb30bd4fdb295f90ed75fa39e9e2e70fb3f753239cf8f1a8e922ec"} Jul 31 23:38:14.002601 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:14.002587 2561 scope.go:117] "RemoveContainer" containerID="b317c413d6eb30bd4fdb295f90ed75fa39e9e2e70fb3f753239cf8f1a8e922ec" Jul 31 23:38:14.003661 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:14.003623 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:38:14.003788 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:14.003703 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:38:14.003788 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:14.003751 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:38:15.009475 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:15.009440 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" event={"ID":"c477bcf3-d54d-4b2a-9c75-79f43a423b01","Type":"ContainerStarted","Data":"052bff9c494b5b184d6762466fd51fa7ace8cf06ca151e203aec62838ad2876f"} Jul 31 23:38:15.461546 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:15.461477 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:38:15.461903 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:15.461877 2561 patch_prober.go:28] interesting pod/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 31 23:38:15.462002 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:15.461949 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" podUID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 31 23:38:16.461228 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:16.461186 2561 patch_prober.go:28] interesting pod/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 31 23:38:16.461702 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:16.461249 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" podUID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 31 23:38:17.461894 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:17.461843 2561 patch_prober.go:28] interesting pod/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 31 23:38:17.462412 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:17.461938 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" podUID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 31 23:38:18.461419 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:18.461379 2561 patch_prober.go:28] interesting pod/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 31 23:38:18.461627 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:18.461441 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" podUID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 31 23:38:19.461865 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:19.461823 2561 patch_prober.go:28] interesting pod/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 31 23:38:19.462369 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:19.461901 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" podUID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 31 23:38:20.465558 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:20.465486 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:38:20.466028 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:20.465936 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:38:20.466806 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:20.466783 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:38:24.731129 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:24.731089 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:38:25.965745 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:25.965693 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:38:33.407560 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:38:33.407513 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:39:04.490695 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:39:04.490661 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:39:07.453710 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:39:07.453673 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:39:16.683458 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:39:16.683377 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:39:35.549742 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:39:35.549703 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:39:44.430869 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:39:44.430833 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:39:45.330631 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:39:45.330599 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:39:56.083593 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:39:56.083552 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:39:56.180315 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:39:56.180276 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:39:56.264243 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:39:56.264148 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:39:56.327105 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:39:56.327059 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:39:56.629661 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:39:56.629616 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:02.442775 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:02.442737 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:02.547689 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:02.547651 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:02.758114 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:02.758019 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h"] Jul 31 23:40:02.758296 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:02.758268 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" podUID="95c885a7-17fb-4483-a517-33f3f886ac23" containerName="istio-proxy" containerID="cri-o://0d9f61afc8526aa20799d7aa41cff426737dc04e0a8718e05ee1fa2a23453608" gracePeriod=30 Jul 31 23:40:02.787806 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:02.787774 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:02.948592 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:02.948555 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:08.027003 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.026934 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:40:08.107589 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.107550 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-workload-socket\") pod \"95c885a7-17fb-4483-a517-33f3f886ac23\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " Jul 31 23:40:08.107589 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.107599 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95c885a7-17fb-4483-a517-33f3f886ac23-istiod-ca-cert\") pod \"95c885a7-17fb-4483-a517-33f3f886ac23\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " Jul 31 23:40:08.107852 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.107623 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-credential-socket\") pod \"95c885a7-17fb-4483-a517-33f3f886ac23\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " Jul 31 23:40:08.107852 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.107643 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95c885a7-17fb-4483-a517-33f3f886ac23-istio-podinfo\") pod \"95c885a7-17fb-4483-a517-33f3f886ac23\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " Jul 31 23:40:08.107852 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.107819 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-istio-data\") pod \"95c885a7-17fb-4483-a517-33f3f886ac23\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " Jul 31 23:40:08.107984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.107883 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9lwr\" (UniqueName: \"kubernetes.io/projected/95c885a7-17fb-4483-a517-33f3f886ac23-kube-api-access-r9lwr\") pod \"95c885a7-17fb-4483-a517-33f3f886ac23\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " Jul 31 23:40:08.107984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.107890 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "95c885a7-17fb-4483-a517-33f3f886ac23" (UID: "95c885a7-17fb-4483-a517-33f3f886ac23"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:40:08.107984 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.107905 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "95c885a7-17fb-4483-a517-33f3f886ac23" (UID: "95c885a7-17fb-4483-a517-33f3f886ac23"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:40:08.108151 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.107986 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-istio-envoy\") pod \"95c885a7-17fb-4483-a517-33f3f886ac23\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " Jul 31 23:40:08.108151 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.108044 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-workload-certs\") pod \"95c885a7-17fb-4483-a517-33f3f886ac23\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " Jul 31 23:40:08.108151 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.108046 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-istio-data" (OuterVolumeSpecName: "istio-data") pod "95c885a7-17fb-4483-a517-33f3f886ac23" (UID: "95c885a7-17fb-4483-a517-33f3f886ac23"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:40:08.108151 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.108076 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95c885a7-17fb-4483-a517-33f3f886ac23-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "95c885a7-17fb-4483-a517-33f3f886ac23" (UID: "95c885a7-17fb-4483-a517-33f3f886ac23"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:40:08.108151 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.108103 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95c885a7-17fb-4483-a517-33f3f886ac23-istio-token\") pod \"95c885a7-17fb-4483-a517-33f3f886ac23\" (UID: \"95c885a7-17fb-4483-a517-33f3f886ac23\") " Jul 31 23:40:08.108399 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.108322 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "95c885a7-17fb-4483-a517-33f3f886ac23" (UID: "95c885a7-17fb-4483-a517-33f3f886ac23"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:40:08.108472 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.108445 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-workload-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:08.108472 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.108467 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-workload-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:08.108620 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.108481 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95c885a7-17fb-4483-a517-33f3f886ac23-istiod-ca-cert\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:08.108620 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.108490 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-credential-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:08.108620 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.108528 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-istio-data\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:08.110079 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.110044 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95c885a7-17fb-4483-a517-33f3f886ac23-kube-api-access-r9lwr" (OuterVolumeSpecName: "kube-api-access-r9lwr") pod "95c885a7-17fb-4483-a517-33f3f886ac23" (UID: "95c885a7-17fb-4483-a517-33f3f886ac23"). InnerVolumeSpecName "kube-api-access-r9lwr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:40:08.110079 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.110056 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/95c885a7-17fb-4483-a517-33f3f886ac23-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "95c885a7-17fb-4483-a517-33f3f886ac23" (UID: "95c885a7-17fb-4483-a517-33f3f886ac23"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 23:40:08.110079 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.110074 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "95c885a7-17fb-4483-a517-33f3f886ac23" (UID: "95c885a7-17fb-4483-a517-33f3f886ac23"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:40:08.110264 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.110132 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95c885a7-17fb-4483-a517-33f3f886ac23-istio-token" (OuterVolumeSpecName: "istio-token") pod "95c885a7-17fb-4483-a517-33f3f886ac23" (UID: "95c885a7-17fb-4483-a517-33f3f886ac23"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:40:08.209093 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.209061 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95c885a7-17fb-4483-a517-33f3f886ac23-istio-envoy\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:08.209093 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.209088 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95c885a7-17fb-4483-a517-33f3f886ac23-istio-token\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:08.209093 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.209099 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95c885a7-17fb-4483-a517-33f3f886ac23-istio-podinfo\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:08.209331 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.209107 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9lwr\" (UniqueName: \"kubernetes.io/projected/95c885a7-17fb-4483-a517-33f3f886ac23-kube-api-access-r9lwr\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:08.646475 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.646442 2561 generic.go:356] "Generic (PLEG): container finished" podID="95c885a7-17fb-4483-a517-33f3f886ac23" containerID="0d9f61afc8526aa20799d7aa41cff426737dc04e0a8718e05ee1fa2a23453608" exitCode=0 Jul 31 23:40:08.646700 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.646490 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" Jul 31 23:40:08.646700 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.646529 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" event={"ID":"95c885a7-17fb-4483-a517-33f3f886ac23","Type":"ContainerDied","Data":"0d9f61afc8526aa20799d7aa41cff426737dc04e0a8718e05ee1fa2a23453608"} Jul 31 23:40:08.646700 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.646590 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h" event={"ID":"95c885a7-17fb-4483-a517-33f3f886ac23","Type":"ContainerDied","Data":"bb6793d11f5609c315047bda9977162c409ddbca55bb6d8b4fb59a660478277c"} Jul 31 23:40:08.646700 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.646604 2561 scope.go:117] "RemoveContainer" containerID="0d9f61afc8526aa20799d7aa41cff426737dc04e0a8718e05ee1fa2a23453608" Jul 31 23:40:08.655808 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.655768 2561 scope.go:117] "RemoveContainer" containerID="45ebf0482266987e70b63fe562efef5dda68c50914f5669a186f6c05f28682f9" Jul 31 23:40:08.663839 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.663817 2561 scope.go:117] "RemoveContainer" containerID="0d9f61afc8526aa20799d7aa41cff426737dc04e0a8718e05ee1fa2a23453608" Jul 31 23:40:08.664070 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:40:08.664051 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d9f61afc8526aa20799d7aa41cff426737dc04e0a8718e05ee1fa2a23453608\": container with ID starting with 0d9f61afc8526aa20799d7aa41cff426737dc04e0a8718e05ee1fa2a23453608 not found: ID does not exist" containerID="0d9f61afc8526aa20799d7aa41cff426737dc04e0a8718e05ee1fa2a23453608" Jul 31 23:40:08.664120 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.664080 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d9f61afc8526aa20799d7aa41cff426737dc04e0a8718e05ee1fa2a23453608"} err="failed to get container status \"0d9f61afc8526aa20799d7aa41cff426737dc04e0a8718e05ee1fa2a23453608\": rpc error: code = NotFound desc = could not find container \"0d9f61afc8526aa20799d7aa41cff426737dc04e0a8718e05ee1fa2a23453608\": container with ID starting with 0d9f61afc8526aa20799d7aa41cff426737dc04e0a8718e05ee1fa2a23453608 not found: ID does not exist" Jul 31 23:40:08.664120 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.664097 2561 scope.go:117] "RemoveContainer" containerID="45ebf0482266987e70b63fe562efef5dda68c50914f5669a186f6c05f28682f9" Jul 31 23:40:08.664305 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:40:08.664287 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45ebf0482266987e70b63fe562efef5dda68c50914f5669a186f6c05f28682f9\": container with ID starting with 45ebf0482266987e70b63fe562efef5dda68c50914f5669a186f6c05f28682f9 not found: ID does not exist" containerID="45ebf0482266987e70b63fe562efef5dda68c50914f5669a186f6c05f28682f9" Jul 31 23:40:08.664346 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.664311 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45ebf0482266987e70b63fe562efef5dda68c50914f5669a186f6c05f28682f9"} err="failed to get container status \"45ebf0482266987e70b63fe562efef5dda68c50914f5669a186f6c05f28682f9\": rpc error: code = NotFound desc = could not find container \"45ebf0482266987e70b63fe562efef5dda68c50914f5669a186f6c05f28682f9\": container with ID starting with 45ebf0482266987e70b63fe562efef5dda68c50914f5669a186f6c05f28682f9 not found: ID does not exist" Jul 31 23:40:08.668729 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.668707 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h"] Jul 31 23:40:08.679333 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:08.679312 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-02002b-openshift-default-67bc58c68b-xts7h"] Jul 31 23:40:10.138288 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:10.138254 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95c885a7-17fb-4483-a517-33f3f886ac23" path="/var/lib/kubelet/pods/95c885a7-17fb-4483-a517-33f3f886ac23/volumes" Jul 31 23:40:13.519614 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:13.519567 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:13.619447 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:13.619403 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:13.720943 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:13.720906 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:14.051469 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:14.051436 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:20.704893 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:20.704856 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:20.804279 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:20.804246 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:20.970130 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:20.970043 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h"] Jul 31 23:40:20.970306 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:20.970281 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" podUID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" containerName="istio-proxy" containerID="cri-o://052bff9c494b5b184d6762466fd51fa7ace8cf06ca151e203aec62838ad2876f" gracePeriod=30 Jul 31 23:40:21.008983 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.008946 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:21.261170 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.261087 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:21.527267 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.527149 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:21.806588 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.806556 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw"] Jul 31 23:40:21.807087 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.807064 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95c885a7-17fb-4483-a517-33f3f886ac23" containerName="istio-proxy" Jul 31 23:40:21.807087 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.807088 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="95c885a7-17fb-4483-a517-33f3f886ac23" containerName="istio-proxy" Jul 31 23:40:21.807266 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.807108 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95c885a7-17fb-4483-a517-33f3f886ac23" containerName="istio-proxy" Jul 31 23:40:21.807266 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.807115 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="95c885a7-17fb-4483-a517-33f3f886ac23" containerName="istio-proxy" Jul 31 23:40:21.807266 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.807185 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="95c885a7-17fb-4483-a517-33f3f886ac23" containerName="istio-proxy" Jul 31 23:40:21.807266 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.807193 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="95c885a7-17fb-4483-a517-33f3f886ac23" containerName="istio-proxy" Jul 31 23:40:21.812033 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.812013 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:21.815424 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.815400 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-4734d6-openshift-default-dockercfg-9vmw5\"" Jul 31 23:40:21.827222 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.827199 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw"] Jul 31 23:40:21.857258 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.857224 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:21.951394 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.951360 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istiod-ca-cert\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:21.951625 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.951417 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpvqb\" (UniqueName: \"kubernetes.io/projected/6cc5416e-9f18-41cc-9b70-0da14e560d7c-kube-api-access-qpvqb\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:21.951625 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.951440 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-data\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:21.951625 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.951462 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-workload-certs\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:21.951625 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.951557 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-workload-socket\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:21.951858 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.951660 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-credential-socket\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:21.951858 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.951712 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-token\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:21.951858 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.951752 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-envoy\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:21.951858 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:21.951784 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-podinfo\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.011083 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.011050 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:22.053340 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.053293 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-token\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.053340 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.053337 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-envoy\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.053661 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.053360 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-podinfo\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.053661 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.053392 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istiod-ca-cert\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.053661 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.053443 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpvqb\" (UniqueName: \"kubernetes.io/projected/6cc5416e-9f18-41cc-9b70-0da14e560d7c-kube-api-access-qpvqb\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.053661 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.053467 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-data\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.053661 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.053497 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-workload-certs\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.053661 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.053625 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-workload-socket\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.053979 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.053693 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-credential-socket\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.053979 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.053945 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-data\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.054084 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.054004 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-workload-certs\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.054225 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.054188 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-workload-socket\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.054298 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.054246 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-credential-socket\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.054353 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.054324 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istiod-ca-cert\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.055759 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.055741 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-envoy\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.056061 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.056043 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-podinfo\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.061076 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.061013 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-token\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.061175 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.061148 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpvqb\" (UniqueName: \"kubernetes.io/projected/6cc5416e-9f18-41cc-9b70-0da14e560d7c-kube-api-access-qpvqb\") pod \"e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.123127 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.123082 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:22.268720 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.268692 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw"] Jul 31 23:40:22.271087 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:40:22.271057 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc5416e_9f18_41cc_9b70_0da14e560d7c.slice/crio-87e77b1e0dd6d3a69c7e3a0635867487e37b06e568bc2575550c597a5ca9ff45 WatchSource:0}: Error finding container 87e77b1e0dd6d3a69c7e3a0635867487e37b06e568bc2575550c597a5ca9ff45: Status 404 returned error can't find the container with id 87e77b1e0dd6d3a69c7e3a0635867487e37b06e568bc2575550c597a5ca9ff45 Jul 31 23:40:22.273368 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.273335 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:40:22.273434 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.273408 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:40:22.273473 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.273462 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:40:22.723730 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.723640 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" event={"ID":"6cc5416e-9f18-41cc-9b70-0da14e560d7c","Type":"ContainerStarted","Data":"5d5dfe59c549a279c35d1dd0efad987f2a85cb47614eda819f8dc2e57c16532d"} Jul 31 23:40:22.723730 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.723693 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" event={"ID":"6cc5416e-9f18-41cc-9b70-0da14e560d7c","Type":"ContainerStarted","Data":"87e77b1e0dd6d3a69c7e3a0635867487e37b06e568bc2575550c597a5ca9ff45"} Jul 31 23:40:22.752283 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:22.752227 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" podStartSLOduration=1.752208963 podStartE2EDuration="1.752208963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:40:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:40:22.750163971 +0000 UTC m=+2579.222608153" watchObservedRunningTime="2026-07-31 23:40:22.752208963 +0000 UTC m=+2579.224653146" Jul 31 23:40:23.123574 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:23.123535 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:23.128826 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:23.128800 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:23.731096 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:23.731061 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:23.732174 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:23.732148 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:24.638039 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:24.637955 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:24.734884 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:24.734852 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:25.759516 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:25.759459 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:26.105798 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.105767 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:26.232598 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.232573 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:40:26.335996 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.335953 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:26.401454 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.401365 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-token\") pod \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " Jul 31 23:40:26.401454 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.401439 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istiod-ca-cert\") pod \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " Jul 31 23:40:26.401695 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.401473 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-workload-socket\") pod \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " Jul 31 23:40:26.401695 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.401562 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-podinfo\") pod \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " Jul 31 23:40:26.401695 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.401599 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-data\") pod \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " Jul 31 23:40:26.401695 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.401656 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-credential-socket\") pod \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " Jul 31 23:40:26.401695 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.401686 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-workload-certs\") pod \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " Jul 31 23:40:26.401943 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.401727 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-envoy\") pod \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " Jul 31 23:40:26.401943 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.401751 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqxpl\" (UniqueName: \"kubernetes.io/projected/c477bcf3-d54d-4b2a-9c75-79f43a423b01-kube-api-access-bqxpl\") pod \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\" (UID: \"c477bcf3-d54d-4b2a-9c75-79f43a423b01\") " Jul 31 23:40:26.401943 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.401848 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c477bcf3-d54d-4b2a-9c75-79f43a423b01" (UID: "c477bcf3-d54d-4b2a-9c75-79f43a423b01"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:40:26.401943 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.401901 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c477bcf3-d54d-4b2a-9c75-79f43a423b01" (UID: "c477bcf3-d54d-4b2a-9c75-79f43a423b01"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:40:26.402142 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.401992 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-data" (OuterVolumeSpecName: "istio-data") pod "c477bcf3-d54d-4b2a-9c75-79f43a423b01" (UID: "c477bcf3-d54d-4b2a-9c75-79f43a423b01"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:40:26.402142 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.402014 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c477bcf3-d54d-4b2a-9c75-79f43a423b01" (UID: "c477bcf3-d54d-4b2a-9c75-79f43a423b01"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:40:26.402142 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.402104 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c477bcf3-d54d-4b2a-9c75-79f43a423b01" (UID: "c477bcf3-d54d-4b2a-9c75-79f43a423b01"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:40:26.402289 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.402262 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-data\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:26.402347 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.402284 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-credential-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:26.402347 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.402300 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-workload-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:26.402347 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.402309 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istiod-ca-cert\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:26.402347 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.402317 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-workload-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:26.403882 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.403859 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-token" (OuterVolumeSpecName: "istio-token") pod "c477bcf3-d54d-4b2a-9c75-79f43a423b01" (UID: "c477bcf3-d54d-4b2a-9c75-79f43a423b01"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:40:26.404032 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.404009 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c477bcf3-d54d-4b2a-9c75-79f43a423b01" (UID: "c477bcf3-d54d-4b2a-9c75-79f43a423b01"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:40:26.404313 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.404296 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c477bcf3-d54d-4b2a-9c75-79f43a423b01" (UID: "c477bcf3-d54d-4b2a-9c75-79f43a423b01"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 23:40:26.404377 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.404306 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c477bcf3-d54d-4b2a-9c75-79f43a423b01-kube-api-access-bqxpl" (OuterVolumeSpecName: "kube-api-access-bqxpl") pod "c477bcf3-d54d-4b2a-9c75-79f43a423b01" (UID: "c477bcf3-d54d-4b2a-9c75-79f43a423b01"). InnerVolumeSpecName "kube-api-access-bqxpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:40:26.426799 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.426768 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:26.502921 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.502887 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-podinfo\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:26.502921 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.502918 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-envoy\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:26.503100 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.502933 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqxpl\" (UniqueName: \"kubernetes.io/projected/c477bcf3-d54d-4b2a-9c75-79f43a423b01-kube-api-access-bqxpl\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:26.503100 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.502947 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c477bcf3-d54d-4b2a-9c75-79f43a423b01-istio-token\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:40:26.749242 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.749150 2561 generic.go:356] "Generic (PLEG): container finished" podID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" containerID="052bff9c494b5b184d6762466fd51fa7ace8cf06ca151e203aec62838ad2876f" exitCode=0 Jul 31 23:40:26.749242 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.749207 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" Jul 31 23:40:26.749488 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.749244 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" event={"ID":"c477bcf3-d54d-4b2a-9c75-79f43a423b01","Type":"ContainerDied","Data":"052bff9c494b5b184d6762466fd51fa7ace8cf06ca151e203aec62838ad2876f"} Jul 31 23:40:26.749488 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.749309 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h" event={"ID":"c477bcf3-d54d-4b2a-9c75-79f43a423b01","Type":"ContainerDied","Data":"6d4e302c305e22e33c7b82b0e1dc52a272799de92b6f88d0db7e8c7cf739436c"} Jul 31 23:40:26.749488 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.749343 2561 scope.go:117] "RemoveContainer" containerID="052bff9c494b5b184d6762466fd51fa7ace8cf06ca151e203aec62838ad2876f" Jul 31 23:40:26.760290 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.760271 2561 scope.go:117] "RemoveContainer" containerID="b317c413d6eb30bd4fdb295f90ed75fa39e9e2e70fb3f753239cf8f1a8e922ec" Jul 31 23:40:26.768597 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.768576 2561 scope.go:117] "RemoveContainer" containerID="052bff9c494b5b184d6762466fd51fa7ace8cf06ca151e203aec62838ad2876f" Jul 31 23:40:26.768871 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:40:26.768853 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"052bff9c494b5b184d6762466fd51fa7ace8cf06ca151e203aec62838ad2876f\": container with ID starting with 052bff9c494b5b184d6762466fd51fa7ace8cf06ca151e203aec62838ad2876f not found: ID does not exist" containerID="052bff9c494b5b184d6762466fd51fa7ace8cf06ca151e203aec62838ad2876f" Jul 31 23:40:26.768916 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.768882 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"052bff9c494b5b184d6762466fd51fa7ace8cf06ca151e203aec62838ad2876f"} err="failed to get container status \"052bff9c494b5b184d6762466fd51fa7ace8cf06ca151e203aec62838ad2876f\": rpc error: code = NotFound desc = could not find container \"052bff9c494b5b184d6762466fd51fa7ace8cf06ca151e203aec62838ad2876f\": container with ID starting with 052bff9c494b5b184d6762466fd51fa7ace8cf06ca151e203aec62838ad2876f not found: ID does not exist" Jul 31 23:40:26.768916 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.768900 2561 scope.go:117] "RemoveContainer" containerID="b317c413d6eb30bd4fdb295f90ed75fa39e9e2e70fb3f753239cf8f1a8e922ec" Jul 31 23:40:26.769156 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:40:26.769136 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b317c413d6eb30bd4fdb295f90ed75fa39e9e2e70fb3f753239cf8f1a8e922ec\": container with ID starting with b317c413d6eb30bd4fdb295f90ed75fa39e9e2e70fb3f753239cf8f1a8e922ec not found: ID does not exist" containerID="b317c413d6eb30bd4fdb295f90ed75fa39e9e2e70fb3f753239cf8f1a8e922ec" Jul 31 23:40:26.769197 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.769163 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b317c413d6eb30bd4fdb295f90ed75fa39e9e2e70fb3f753239cf8f1a8e922ec"} err="failed to get container status \"b317c413d6eb30bd4fdb295f90ed75fa39e9e2e70fb3f753239cf8f1a8e922ec\": rpc error: code = NotFound desc = could not find container \"b317c413d6eb30bd4fdb295f90ed75fa39e9e2e70fb3f753239cf8f1a8e922ec\": container with ID starting with b317c413d6eb30bd4fdb295f90ed75fa39e9e2e70fb3f753239cf8f1a8e922ec not found: ID does not exist" Jul 31 23:40:26.776215 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.776195 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h"] Jul 31 23:40:26.782055 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:26.782029 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-3c3302-openshift-default-84d8788b4-rfs7h"] Jul 31 23:40:28.132886 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:28.132845 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" path="/var/lib/kubelet/pods/c477bcf3-d54d-4b2a-9c75-79f43a423b01/volumes" Jul 31 23:40:37.813111 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:37.813078 2561 generic.go:356] "Generic (PLEG): container finished" podID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" containerID="5d5dfe59c549a279c35d1dd0efad987f2a85cb47614eda819f8dc2e57c16532d" exitCode=0 Jul 31 23:40:37.813598 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:37.813155 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" event={"ID":"6cc5416e-9f18-41cc-9b70-0da14e560d7c","Type":"ContainerDied","Data":"5d5dfe59c549a279c35d1dd0efad987f2a85cb47614eda819f8dc2e57c16532d"} Jul 31 23:40:37.813598 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:37.813592 2561 scope.go:117] "RemoveContainer" containerID="5d5dfe59c549a279c35d1dd0efad987f2a85cb47614eda819f8dc2e57c16532d" Jul 31 23:40:37.814709 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:37.814680 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:40:37.814820 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:37.814745 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:40:37.814820 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:37.814775 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:40:38.731003 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:38.730956 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:38.819840 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:38.819808 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" event={"ID":"6cc5416e-9f18-41cc-9b70-0da14e560d7c","Type":"ContainerStarted","Data":"3ef1ba76a44844dcc6eb5f212144b09b10136d1512670a32d9b53b34bfb98f64"} Jul 31 23:40:39.123565 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:39.123533 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:39.124655 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:39.124627 2561 patch_prober.go:28] interesting pod/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 31 23:40:39.124796 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:39.124688 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" podUID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 31 23:40:40.123475 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:40.123438 2561 patch_prober.go:28] interesting pod/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 31 23:40:40.123903 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:40.123492 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" podUID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 31 23:40:41.124410 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:41.124377 2561 patch_prober.go:28] interesting pod/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 31 23:40:41.124863 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:41.124431 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" podUID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 31 23:40:42.123840 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:42.123799 2561 patch_prober.go:28] interesting pod/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 31 23:40:42.124027 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:42.123854 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" podUID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 31 23:40:43.124025 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:43.123978 2561 patch_prober.go:28] interesting pod/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 31 23:40:43.124519 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:43.124043 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" podUID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 31 23:40:44.133440 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:44.133412 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:44.133851 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:44.133749 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:44.134583 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:44.134561 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:40:49.572253 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:49.572173 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:50.268834 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.268794 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm"] Jul 31 23:40:50.269292 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.269274 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" containerName="istio-proxy" Jul 31 23:40:50.269392 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.269294 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" containerName="istio-proxy" Jul 31 23:40:50.269392 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.269322 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" containerName="istio-proxy" Jul 31 23:40:50.269392 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.269330 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" containerName="istio-proxy" Jul 31 23:40:50.269594 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.269425 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" containerName="istio-proxy" Jul 31 23:40:50.269594 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.269444 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c477bcf3-d54d-4b2a-9c75-79f43a423b01" containerName="istio-proxy" Jul 31 23:40:50.272803 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.272782 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.275309 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.275290 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-f7d39b-openshift-default-dockercfg-sffdf\"" Jul 31 23:40:50.284444 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.284417 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm"] Jul 31 23:40:50.372271 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.372236 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-workload-certs\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.372468 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.372298 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-envoy\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.372468 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.372331 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-workload-socket\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.372468 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.372351 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-podinfo\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.372468 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.372384 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9htdl\" (UniqueName: \"kubernetes.io/projected/9d1e288a-8154-4e1e-bf7e-d7604b831916-kube-api-access-9htdl\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.372468 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.372411 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-credential-socket\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.372774 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.372490 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-data\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.372774 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.372541 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-token\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.372774 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.372570 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d1e288a-8154-4e1e-bf7e-d7604b831916-istiod-ca-cert\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.473868 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.473831 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-workload-certs\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.474062 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.473880 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-envoy\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.474062 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.473903 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-workload-socket\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.474062 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.473922 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-podinfo\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.474062 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.473978 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9htdl\" (UniqueName: \"kubernetes.io/projected/9d1e288a-8154-4e1e-bf7e-d7604b831916-kube-api-access-9htdl\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.474062 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.474026 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-credential-socket\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.474351 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.474120 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-data\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.474351 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.474155 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-token\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.474351 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.474200 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d1e288a-8154-4e1e-bf7e-d7604b831916-istiod-ca-cert\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.474351 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.474279 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-workload-certs\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.474584 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.474399 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-credential-socket\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.474584 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.474493 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-workload-socket\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.474584 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.474553 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-data\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.474995 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.474971 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d1e288a-8154-4e1e-bf7e-d7604b831916-istiod-ca-cert\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.476326 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.476308 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-envoy\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.476377 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.476352 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-podinfo\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.482975 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.482952 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-token\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.483163 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.483146 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9htdl\" (UniqueName: \"kubernetes.io/projected/9d1e288a-8154-4e1e-bf7e-d7604b831916-kube-api-access-9htdl\") pod \"e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.583307 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.583265 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:50.720913 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.720888 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm"] Jul 31 23:40:50.722951 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:40:50.722925 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d1e288a_8154_4e1e_bf7e_d7604b831916.slice/crio-a31e8003ad7f56beea3b8329131bcde8925e192a06b27161089e9d96d88c9e41 WatchSource:0}: Error finding container a31e8003ad7f56beea3b8329131bcde8925e192a06b27161089e9d96d88c9e41: Status 404 returned error can't find the container with id a31e8003ad7f56beea3b8329131bcde8925e192a06b27161089e9d96d88c9e41 Jul 31 23:40:50.725138 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.725111 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:40:50.725218 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.725170 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:40:50.725218 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.725205 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:40:50.834288 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.834194 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:50.889052 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.889012 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" event={"ID":"9d1e288a-8154-4e1e-bf7e-d7604b831916","Type":"ContainerStarted","Data":"b3f08fa3ee92da2f1b35d24975fc094782e9f60299c19e8b5a98f99ecc8d48b1"} Jul 31 23:40:50.889225 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.889080 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" event={"ID":"9d1e288a-8154-4e1e-bf7e-d7604b831916","Type":"ContainerStarted","Data":"a31e8003ad7f56beea3b8329131bcde8925e192a06b27161089e9d96d88c9e41"} Jul 31 23:40:50.908813 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:50.908755 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" podStartSLOduration=0.908738408 podStartE2EDuration="908.738408ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:40:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:40:50.907268012 +0000 UTC m=+2607.379712188" watchObservedRunningTime="2026-07-31 23:40:50.908738408 +0000 UTC m=+2607.381182590" Jul 31 23:40:51.584213 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:51.584177 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:51.589221 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:51.589195 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:51.894698 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:51.894609 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:51.895756 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:51.895737 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:40:52.753243 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:52.753206 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:52.898880 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:52.898844 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:53.917867 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:53.917791 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:54.024988 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:54.024944 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:54.403391 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:54.403356 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:40:54.498849 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:40:54.498811 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:41:07.343895 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:07.342767 2561 patch_prober.go:28] interesting pod/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": read tcp 10.134.0.2:50748->10.134.0.55:15021: read: connection reset by peer" start-of-body= Jul 31 23:41:07.343895 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:07.342836 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" podUID="9d1e288a-8154-4e1e-bf7e-d7604b831916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": read tcp 10.134.0.2:50748->10.134.0.55:15021: read: connection reset by peer" Jul 31 23:41:08.003410 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:08.003365 2561 generic.go:356] "Generic (PLEG): container finished" podID="9d1e288a-8154-4e1e-bf7e-d7604b831916" containerID="b3f08fa3ee92da2f1b35d24975fc094782e9f60299c19e8b5a98f99ecc8d48b1" exitCode=0 Jul 31 23:41:08.003658 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:08.003440 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" event={"ID":"9d1e288a-8154-4e1e-bf7e-d7604b831916","Type":"ContainerDied","Data":"b3f08fa3ee92da2f1b35d24975fc094782e9f60299c19e8b5a98f99ecc8d48b1"} Jul 31 23:41:08.004001 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:08.003983 2561 scope.go:117] "RemoveContainer" containerID="b3f08fa3ee92da2f1b35d24975fc094782e9f60299c19e8b5a98f99ecc8d48b1" Jul 31 23:41:08.005166 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:08.005136 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:41:08.005234 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:08.005220 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:41:08.005272 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:08.005256 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:41:09.010746 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:09.010708 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" event={"ID":"9d1e288a-8154-4e1e-bf7e-d7604b831916","Type":"ContainerStarted","Data":"60aec8f044653c812b84d36ae711646d0e8deb972295c4ecf045c7db9ad69089"} Jul 31 23:41:09.584432 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:09.584390 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:41:09.584718 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:09.584614 2561 patch_prober.go:28] interesting pod/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 31 23:41:09.584718 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:09.584662 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" podUID="9d1e288a-8154-4e1e-bf7e-d7604b831916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 31 23:41:10.583977 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:10.583930 2561 patch_prober.go:28] interesting pod/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 31 23:41:10.584385 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:10.584014 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" podUID="9d1e288a-8154-4e1e-bf7e-d7604b831916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 31 23:41:11.584584 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:11.584541 2561 patch_prober.go:28] interesting pod/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 31 23:41:11.584985 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:11.584615 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" podUID="9d1e288a-8154-4e1e-bf7e-d7604b831916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 31 23:41:12.583958 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:12.583918 2561 patch_prober.go:28] interesting pod/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 31 23:41:12.584144 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:12.583985 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" podUID="9d1e288a-8154-4e1e-bf7e-d7604b831916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 31 23:41:13.584335 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:13.584296 2561 patch_prober.go:28] interesting pod/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 31 23:41:13.584764 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:13.584363 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" podUID="9d1e288a-8154-4e1e-bf7e-d7604b831916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 31 23:41:14.587352 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:14.587319 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:41:14.587832 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:14.587617 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:41:14.588277 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:14.588261 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:41:17.700147 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:17.700091 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:41:17.965467 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:17.965379 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:41:35.994292 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:35.994246 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:41:36.092051 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:36.092016 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:41:36.149257 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:36.149221 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:41:36.507723 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:36.507689 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:41:42.114343 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:42.114305 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:41:42.214539 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:42.214472 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:41:42.348890 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:42.348846 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw"] Jul 31 23:41:42.349205 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:42.349151 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" podUID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" containerName="istio-proxy" containerID="cri-o://3ef1ba76a44844dcc6eb5f212144b09b10136d1512670a32d9b53b34bfb98f64" gracePeriod=30 Jul 31 23:41:42.372740 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:42.372652 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:41:42.623996 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:42.623896 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:41:44.135811 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:44.135773 2561 patch_prober.go:28] interesting pod/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 31 23:41:44.136217 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:44.135848 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" podUID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 31 23:41:47.610238 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.610207 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:41:47.673388 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.673283 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-data\") pod \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " Jul 31 23:41:47.673388 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.673323 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-podinfo\") pod \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " Jul 31 23:41:47.673388 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.673348 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-credential-socket\") pod \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " Jul 31 23:41:47.673388 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.673380 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-token\") pod \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " Jul 31 23:41:47.673775 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.673404 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istiod-ca-cert\") pod \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " Jul 31 23:41:47.673775 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.673470 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-workload-socket\") pod \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " Jul 31 23:41:47.673775 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.673538 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpvqb\" (UniqueName: \"kubernetes.io/projected/6cc5416e-9f18-41cc-9b70-0da14e560d7c-kube-api-access-qpvqb\") pod \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " Jul 31 23:41:47.673775 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.673594 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-workload-certs\") pod \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " Jul 31 23:41:47.673775 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.673624 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-envoy\") pod \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\" (UID: \"6cc5416e-9f18-41cc-9b70-0da14e560d7c\") " Jul 31 23:41:47.673775 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.673658 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6cc5416e-9f18-41cc-9b70-0da14e560d7c" (UID: "6cc5416e-9f18-41cc-9b70-0da14e560d7c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:41:47.673775 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.673703 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-data" (OuterVolumeSpecName: "istio-data") pod "6cc5416e-9f18-41cc-9b70-0da14e560d7c" (UID: "6cc5416e-9f18-41cc-9b70-0da14e560d7c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:41:47.674139 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.673855 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6cc5416e-9f18-41cc-9b70-0da14e560d7c" (UID: "6cc5416e-9f18-41cc-9b70-0da14e560d7c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:41:47.674139 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.673877 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6cc5416e-9f18-41cc-9b70-0da14e560d7c" (UID: "6cc5416e-9f18-41cc-9b70-0da14e560d7c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:41:47.674139 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.674017 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-workload-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:41:47.674139 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.674066 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-data\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:41:47.674139 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.674081 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-credential-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:41:47.674139 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.674094 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-workload-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:41:47.674444 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.674223 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6cc5416e-9f18-41cc-9b70-0da14e560d7c" (UID: "6cc5416e-9f18-41cc-9b70-0da14e560d7c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:41:47.675794 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.675757 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6cc5416e-9f18-41cc-9b70-0da14e560d7c" (UID: "6cc5416e-9f18-41cc-9b70-0da14e560d7c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 23:41:47.675914 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.675858 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6cc5416e-9f18-41cc-9b70-0da14e560d7c" (UID: "6cc5416e-9f18-41cc-9b70-0da14e560d7c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:41:47.675914 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.675891 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cc5416e-9f18-41cc-9b70-0da14e560d7c-kube-api-access-qpvqb" (OuterVolumeSpecName: "kube-api-access-qpvqb") pod "6cc5416e-9f18-41cc-9b70-0da14e560d7c" (UID: "6cc5416e-9f18-41cc-9b70-0da14e560d7c"). InnerVolumeSpecName "kube-api-access-qpvqb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:41:47.675987 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.675915 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-token" (OuterVolumeSpecName: "istio-token") pod "6cc5416e-9f18-41cc-9b70-0da14e560d7c" (UID: "6cc5416e-9f18-41cc-9b70-0da14e560d7c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:41:47.775059 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.775017 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-envoy\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:41:47.775059 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.775052 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-podinfo\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:41:47.775059 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.775061 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istio-token\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:41:47.775059 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.775071 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cc5416e-9f18-41cc-9b70-0da14e560d7c-istiod-ca-cert\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:41:47.775331 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:47.775079 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpvqb\" (UniqueName: \"kubernetes.io/projected/6cc5416e-9f18-41cc-9b70-0da14e560d7c-kube-api-access-qpvqb\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:41:48.217405 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:48.217371 2561 generic.go:356] "Generic (PLEG): container finished" podID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" containerID="3ef1ba76a44844dcc6eb5f212144b09b10136d1512670a32d9b53b34bfb98f64" exitCode=0 Jul 31 23:41:48.217600 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:48.217446 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" Jul 31 23:41:48.217600 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:48.217453 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" event={"ID":"6cc5416e-9f18-41cc-9b70-0da14e560d7c","Type":"ContainerDied","Data":"3ef1ba76a44844dcc6eb5f212144b09b10136d1512670a32d9b53b34bfb98f64"} Jul 31 23:41:48.217600 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:48.217564 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw" event={"ID":"6cc5416e-9f18-41cc-9b70-0da14e560d7c","Type":"ContainerDied","Data":"87e77b1e0dd6d3a69c7e3a0635867487e37b06e568bc2575550c597a5ca9ff45"} Jul 31 23:41:48.217600 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:48.217597 2561 scope.go:117] "RemoveContainer" containerID="3ef1ba76a44844dcc6eb5f212144b09b10136d1512670a32d9b53b34bfb98f64" Jul 31 23:41:48.229994 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:48.229906 2561 scope.go:117] "RemoveContainer" containerID="5d5dfe59c549a279c35d1dd0efad987f2a85cb47614eda819f8dc2e57c16532d" Jul 31 23:41:48.234905 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:48.234877 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw"] Jul 31 23:41:48.240343 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:48.240321 2561 scope.go:117] "RemoveContainer" containerID="3ef1ba76a44844dcc6eb5f212144b09b10136d1512670a32d9b53b34bfb98f64" Jul 31 23:41:48.240469 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:48.240447 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-4734d6-openshift-default-8689b9f5f6-zbgvw"] Jul 31 23:41:48.240674 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:41:48.240654 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ef1ba76a44844dcc6eb5f212144b09b10136d1512670a32d9b53b34bfb98f64\": container with ID starting with 3ef1ba76a44844dcc6eb5f212144b09b10136d1512670a32d9b53b34bfb98f64 not found: ID does not exist" containerID="3ef1ba76a44844dcc6eb5f212144b09b10136d1512670a32d9b53b34bfb98f64" Jul 31 23:41:48.240731 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:48.240683 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ef1ba76a44844dcc6eb5f212144b09b10136d1512670a32d9b53b34bfb98f64"} err="failed to get container status \"3ef1ba76a44844dcc6eb5f212144b09b10136d1512670a32d9b53b34bfb98f64\": rpc error: code = NotFound desc = could not find container \"3ef1ba76a44844dcc6eb5f212144b09b10136d1512670a32d9b53b34bfb98f64\": container with ID starting with 3ef1ba76a44844dcc6eb5f212144b09b10136d1512670a32d9b53b34bfb98f64 not found: ID does not exist" Jul 31 23:41:48.240731 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:48.240702 2561 scope.go:117] "RemoveContainer" containerID="5d5dfe59c549a279c35d1dd0efad987f2a85cb47614eda819f8dc2e57c16532d" Jul 31 23:41:48.240953 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:41:48.240935 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d5dfe59c549a279c35d1dd0efad987f2a85cb47614eda819f8dc2e57c16532d\": container with ID starting with 5d5dfe59c549a279c35d1dd0efad987f2a85cb47614eda819f8dc2e57c16532d not found: ID does not exist" containerID="5d5dfe59c549a279c35d1dd0efad987f2a85cb47614eda819f8dc2e57c16532d" Jul 31 23:41:48.240992 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:48.240958 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d5dfe59c549a279c35d1dd0efad987f2a85cb47614eda819f8dc2e57c16532d"} err="failed to get container status \"5d5dfe59c549a279c35d1dd0efad987f2a85cb47614eda819f8dc2e57c16532d\": rpc error: code = NotFound desc = could not find container \"5d5dfe59c549a279c35d1dd0efad987f2a85cb47614eda819f8dc2e57c16532d\": container with ID starting with 5d5dfe59c549a279c35d1dd0efad987f2a85cb47614eda819f8dc2e57c16532d not found: ID does not exist" Jul 31 23:41:50.133614 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:50.133577 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" path="/var/lib/kubelet/pods/6cc5416e-9f18-41cc-9b70-0da14e560d7c/volumes" Jul 31 23:41:58.191950 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:58.191913 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:41:58.255865 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:58.255826 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:41:58.317085 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:58.317048 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:41:58.385234 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:58.385194 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:41:58.729813 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:41:58.729781 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:04.677244 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:04.677196 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:04.737757 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:04.737718 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:04.908238 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:04.908200 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm"] Jul 31 23:42:04.908474 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:04.908449 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" podUID="9d1e288a-8154-4e1e-bf7e-d7604b831916" containerName="istio-proxy" containerID="cri-o://60aec8f044653c812b84d36ae711646d0e8deb972295c4ecf045c7db9ad69089" gracePeriod=30 Jul 31 23:42:04.925269 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:04.925233 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:05.140080 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.140046 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:05.461481 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.461368 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:05.720192 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.720087 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck"] Jul 31 23:42:05.720727 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.720707 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" containerName="istio-proxy" Jul 31 23:42:05.720727 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.720726 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" containerName="istio-proxy" Jul 31 23:42:05.720862 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.720743 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" containerName="istio-proxy" Jul 31 23:42:05.720862 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.720749 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" containerName="istio-proxy" Jul 31 23:42:05.720862 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.720822 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" containerName="istio-proxy" Jul 31 23:42:05.720862 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.720832 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cc5416e-9f18-41cc-9b70-0da14e560d7c" containerName="istio-proxy" Jul 31 23:42:05.723993 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.723973 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.726584 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.726564 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-99d0e4-openshift-default-dockercfg-p8wnv\"" Jul 31 23:42:05.736114 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.736088 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck"] Jul 31 23:42:05.750957 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.750922 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:05.853443 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.853409 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-envoy\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.853609 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.853450 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-workload-socket\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.853609 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.853542 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz6g2\" (UniqueName: \"kubernetes.io/projected/751e6f22-f48a-476f-bbea-570bd79d0d2e-kube-api-access-cz6g2\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.853705 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.853637 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-podinfo\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.853705 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.853670 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-workload-certs\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.853780 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.853712 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-data\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.853780 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.853752 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-token\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.853842 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.853784 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-credential-socket\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.853842 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.853802 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/751e6f22-f48a-476f-bbea-570bd79d0d2e-istiod-ca-cert\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.954567 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.954523 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-podinfo\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.954567 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.954572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-workload-certs\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.954829 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.954597 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-data\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.954829 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.954628 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-token\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.954829 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.954654 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-credential-socket\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.954829 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.954670 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/751e6f22-f48a-476f-bbea-570bd79d0d2e-istiod-ca-cert\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.954829 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.954709 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-envoy\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.954829 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.954742 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-workload-socket\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.954829 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.954781 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz6g2\" (UniqueName: \"kubernetes.io/projected/751e6f22-f48a-476f-bbea-570bd79d0d2e-kube-api-access-cz6g2\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.955177 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.955089 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-workload-certs\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.955177 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.955115 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-data\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.955363 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.955345 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-workload-socket\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.955439 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.955383 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-credential-socket\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.955484 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.955454 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/751e6f22-f48a-476f-bbea-570bd79d0d2e-istiod-ca-cert\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.957412 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.957384 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-envoy\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.957563 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.957545 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-podinfo\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.962412 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.962386 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-token\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:05.962549 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:05.962533 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz6g2\" (UniqueName: \"kubernetes.io/projected/751e6f22-f48a-476f-bbea-570bd79d0d2e-kube-api-access-cz6g2\") pod \"e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:06.035698 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:06.035588 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:06.035867 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:06.035736 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:06.180777 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:06.180714 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck"] Jul 31 23:42:06.183746 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:42:06.183718 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod751e6f22_f48a_476f_bbea_570bd79d0d2e.slice/crio-409be6aa897dfaeaf8a748789e1d294e0145600fa87913c0bc805a493bf058fd WatchSource:0}: Error finding container 409be6aa897dfaeaf8a748789e1d294e0145600fa87913c0bc805a493bf058fd: Status 404 returned error can't find the container with id 409be6aa897dfaeaf8a748789e1d294e0145600fa87913c0bc805a493bf058fd Jul 31 23:42:06.185950 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:06.185919 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:42:06.186038 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:06.185988 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:42:06.186038 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:06.186017 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:42:06.314693 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:06.314652 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" event={"ID":"751e6f22-f48a-476f-bbea-570bd79d0d2e","Type":"ContainerStarted","Data":"776083e0411b4b379981bad9264a31ff99728057ffb5a9d75edb58e46c5b88c9"} Jul 31 23:42:06.314892 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:06.314707 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" event={"ID":"751e6f22-f48a-476f-bbea-570bd79d0d2e","Type":"ContainerStarted","Data":"409be6aa897dfaeaf8a748789e1d294e0145600fa87913c0bc805a493bf058fd"} Jul 31 23:42:06.335005 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:06.334953 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" podStartSLOduration=1.334937306 podStartE2EDuration="1.334937306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:42:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:42:06.334560898 +0000 UTC m=+2682.807005081" watchObservedRunningTime="2026-07-31 23:42:06.334937306 +0000 UTC m=+2682.807381488" Jul 31 23:42:07.036411 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:07.036372 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:07.041118 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:07.041090 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:07.320532 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:07.320491 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:07.321407 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:07.321386 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:08.261837 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:08.261801 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:08.361735 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:08.361702 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:09.389759 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:09.389728 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:09.872342 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:09.872273 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:10.058638 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.058594 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:10.164528 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.164472 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:42:10.197861 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.197823 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9htdl\" (UniqueName: \"kubernetes.io/projected/9d1e288a-8154-4e1e-bf7e-d7604b831916-kube-api-access-9htdl\") pod \"9d1e288a-8154-4e1e-bf7e-d7604b831916\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " Jul 31 23:42:10.198049 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.197886 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-envoy\") pod \"9d1e288a-8154-4e1e-bf7e-d7604b831916\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " Jul 31 23:42:10.198049 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.197930 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-data\") pod \"9d1e288a-8154-4e1e-bf7e-d7604b831916\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " Jul 31 23:42:10.198049 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.197959 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-token\") pod \"9d1e288a-8154-4e1e-bf7e-d7604b831916\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " Jul 31 23:42:10.198049 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.197980 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d1e288a-8154-4e1e-bf7e-d7604b831916-istiod-ca-cert\") pod \"9d1e288a-8154-4e1e-bf7e-d7604b831916\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " Jul 31 23:42:10.198049 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.198040 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-workload-certs\") pod \"9d1e288a-8154-4e1e-bf7e-d7604b831916\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " Jul 31 23:42:10.198317 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.198104 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-podinfo\") pod \"9d1e288a-8154-4e1e-bf7e-d7604b831916\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " Jul 31 23:42:10.198317 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.198141 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-workload-socket\") pod \"9d1e288a-8154-4e1e-bf7e-d7604b831916\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " Jul 31 23:42:10.198317 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.198177 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-credential-socket\") pod \"9d1e288a-8154-4e1e-bf7e-d7604b831916\" (UID: \"9d1e288a-8154-4e1e-bf7e-d7604b831916\") " Jul 31 23:42:10.199008 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.198747 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d1e288a-8154-4e1e-bf7e-d7604b831916" (UID: "9d1e288a-8154-4e1e-bf7e-d7604b831916"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:42:10.199008 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.198921 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d1e288a-8154-4e1e-bf7e-d7604b831916" (UID: "9d1e288a-8154-4e1e-bf7e-d7604b831916"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:42:10.199008 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.198942 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d1e288a-8154-4e1e-bf7e-d7604b831916" (UID: "9d1e288a-8154-4e1e-bf7e-d7604b831916"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:42:10.199008 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.198985 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d1e288a-8154-4e1e-bf7e-d7604b831916" (UID: "9d1e288a-8154-4e1e-bf7e-d7604b831916"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:42:10.199566 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.199522 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d1e288a-8154-4e1e-bf7e-d7604b831916-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d1e288a-8154-4e1e-bf7e-d7604b831916" (UID: "9d1e288a-8154-4e1e-bf7e-d7604b831916"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:42:10.203090 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.203054 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d1e288a-8154-4e1e-bf7e-d7604b831916" (UID: "9d1e288a-8154-4e1e-bf7e-d7604b831916"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:42:10.203584 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.203539 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d1e288a-8154-4e1e-bf7e-d7604b831916" (UID: "9d1e288a-8154-4e1e-bf7e-d7604b831916"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 23:42:10.204238 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.204198 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d1e288a-8154-4e1e-bf7e-d7604b831916-kube-api-access-9htdl" (OuterVolumeSpecName: "kube-api-access-9htdl") pod "9d1e288a-8154-4e1e-bf7e-d7604b831916" (UID: "9d1e288a-8154-4e1e-bf7e-d7604b831916"). InnerVolumeSpecName "kube-api-access-9htdl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:42:10.205297 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.205261 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d1e288a-8154-4e1e-bf7e-d7604b831916" (UID: "9d1e288a-8154-4e1e-bf7e-d7604b831916"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:42:10.299701 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.299664 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-workload-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:42:10.299701 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.299695 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-podinfo\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:42:10.299701 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.299704 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-workload-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:42:10.299936 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.299713 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-credential-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:42:10.299936 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.299724 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9htdl\" (UniqueName: \"kubernetes.io/projected/9d1e288a-8154-4e1e-bf7e-d7604b831916-kube-api-access-9htdl\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:42:10.299936 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.299732 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-envoy\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:42:10.299936 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.299740 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-data\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:42:10.299936 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.299747 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d1e288a-8154-4e1e-bf7e-d7604b831916-istio-token\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:42:10.299936 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.299754 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d1e288a-8154-4e1e-bf7e-d7604b831916-istiod-ca-cert\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:42:10.337875 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.337842 2561 generic.go:356] "Generic (PLEG): container finished" podID="9d1e288a-8154-4e1e-bf7e-d7604b831916" containerID="60aec8f044653c812b84d36ae711646d0e8deb972295c4ecf045c7db9ad69089" exitCode=0 Jul 31 23:42:10.338081 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.337900 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" Jul 31 23:42:10.338081 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.337927 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" event={"ID":"9d1e288a-8154-4e1e-bf7e-d7604b831916","Type":"ContainerDied","Data":"60aec8f044653c812b84d36ae711646d0e8deb972295c4ecf045c7db9ad69089"} Jul 31 23:42:10.338081 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.337982 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm" event={"ID":"9d1e288a-8154-4e1e-bf7e-d7604b831916","Type":"ContainerDied","Data":"a31e8003ad7f56beea3b8329131bcde8925e192a06b27161089e9d96d88c9e41"} Jul 31 23:42:10.338081 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.338005 2561 scope.go:117] "RemoveContainer" containerID="60aec8f044653c812b84d36ae711646d0e8deb972295c4ecf045c7db9ad69089" Jul 31 23:42:10.348581 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.348563 2561 scope.go:117] "RemoveContainer" containerID="b3f08fa3ee92da2f1b35d24975fc094782e9f60299c19e8b5a98f99ecc8d48b1" Jul 31 23:42:10.357517 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.357469 2561 scope.go:117] "RemoveContainer" containerID="60aec8f044653c812b84d36ae711646d0e8deb972295c4ecf045c7db9ad69089" Jul 31 23:42:10.357983 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:42:10.357912 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60aec8f044653c812b84d36ae711646d0e8deb972295c4ecf045c7db9ad69089\": container with ID starting with 60aec8f044653c812b84d36ae711646d0e8deb972295c4ecf045c7db9ad69089 not found: ID does not exist" containerID="60aec8f044653c812b84d36ae711646d0e8deb972295c4ecf045c7db9ad69089" Jul 31 23:42:10.357983 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.357959 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60aec8f044653c812b84d36ae711646d0e8deb972295c4ecf045c7db9ad69089"} err="failed to get container status \"60aec8f044653c812b84d36ae711646d0e8deb972295c4ecf045c7db9ad69089\": rpc error: code = NotFound desc = could not find container \"60aec8f044653c812b84d36ae711646d0e8deb972295c4ecf045c7db9ad69089\": container with ID starting with 60aec8f044653c812b84d36ae711646d0e8deb972295c4ecf045c7db9ad69089 not found: ID does not exist" Jul 31 23:42:10.358236 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.357986 2561 scope.go:117] "RemoveContainer" containerID="b3f08fa3ee92da2f1b35d24975fc094782e9f60299c19e8b5a98f99ecc8d48b1" Jul 31 23:42:10.358300 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:42:10.358269 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3f08fa3ee92da2f1b35d24975fc094782e9f60299c19e8b5a98f99ecc8d48b1\": container with ID starting with b3f08fa3ee92da2f1b35d24975fc094782e9f60299c19e8b5a98f99ecc8d48b1 not found: ID does not exist" containerID="b3f08fa3ee92da2f1b35d24975fc094782e9f60299c19e8b5a98f99ecc8d48b1" Jul 31 23:42:10.358356 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.358303 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3f08fa3ee92da2f1b35d24975fc094782e9f60299c19e8b5a98f99ecc8d48b1"} err="failed to get container status \"b3f08fa3ee92da2f1b35d24975fc094782e9f60299c19e8b5a98f99ecc8d48b1\": rpc error: code = NotFound desc = could not find container \"b3f08fa3ee92da2f1b35d24975fc094782e9f60299c19e8b5a98f99ecc8d48b1\": container with ID starting with b3f08fa3ee92da2f1b35d24975fc094782e9f60299c19e8b5a98f99ecc8d48b1 not found: ID does not exist" Jul 31 23:42:10.361053 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.361031 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm"] Jul 31 23:42:10.364248 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:10.364227 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-f7d39b-openshift-default-886bdc4fb-fn8xm"] Jul 31 23:42:12.133308 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:12.133271 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d1e288a-8154-4e1e-bf7e-d7604b831916" path="/var/lib/kubelet/pods/9d1e288a-8154-4e1e-bf7e-d7604b831916/volumes" Jul 31 23:42:22.322792 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:22.322705 2561 patch_prober.go:28] interesting pod/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 31 23:42:22.323195 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:22.322780 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" podUID="751e6f22-f48a-476f-bbea-570bd79d0d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 31 23:42:22.409360 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:22.409275 2561 generic.go:356] "Generic (PLEG): container finished" podID="751e6f22-f48a-476f-bbea-570bd79d0d2e" containerID="776083e0411b4b379981bad9264a31ff99728057ffb5a9d75edb58e46c5b88c9" exitCode=0 Jul 31 23:42:22.409545 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:22.409348 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" event={"ID":"751e6f22-f48a-476f-bbea-570bd79d0d2e","Type":"ContainerDied","Data":"776083e0411b4b379981bad9264a31ff99728057ffb5a9d75edb58e46c5b88c9"} Jul 31 23:42:22.409885 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:22.409867 2561 scope.go:117] "RemoveContainer" containerID="776083e0411b4b379981bad9264a31ff99728057ffb5a9d75edb58e46c5b88c9" Jul 31 23:42:22.411010 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:22.410982 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:42:22.411101 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:22.411049 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:42:22.411101 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:22.411081 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:42:23.416294 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:23.416257 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" event={"ID":"751e6f22-f48a-476f-bbea-570bd79d0d2e","Type":"ContainerStarted","Data":"c357b07ab2f7e9e4f433f9123c98f7bbc875c4525cf65d43593a43a471385c60"} Jul 31 23:42:24.036770 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:24.036731 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:24.037088 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:24.037048 2561 patch_prober.go:28] interesting pod/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 31 23:42:24.037191 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:24.037155 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" podUID="751e6f22-f48a-476f-bbea-570bd79d0d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 31 23:42:25.036996 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:25.036956 2561 patch_prober.go:28] interesting pod/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 31 23:42:25.037615 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:25.037580 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" podUID="751e6f22-f48a-476f-bbea-570bd79d0d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 31 23:42:26.036988 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:26.036947 2561 patch_prober.go:28] interesting pod/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 31 23:42:26.037331 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:26.037293 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" podUID="751e6f22-f48a-476f-bbea-570bd79d0d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 31 23:42:27.036815 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:27.036766 2561 patch_prober.go:28] interesting pod/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 31 23:42:27.037049 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:27.036846 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" podUID="751e6f22-f48a-476f-bbea-570bd79d0d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 31 23:42:28.036807 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:28.036758 2561 patch_prober.go:28] interesting pod/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 31 23:42:28.037314 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:28.036839 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" podUID="751e6f22-f48a-476f-bbea-570bd79d0d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 31 23:42:29.040800 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:29.040769 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:29.041187 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:29.041125 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:29.041792 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:29.041772 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:42:33.111463 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.111420 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:33.439179 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.439077 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:33.698021 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.697939 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc"] Jul 31 23:42:33.698429 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.698409 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d1e288a-8154-4e1e-bf7e-d7604b831916" containerName="istio-proxy" Jul 31 23:42:33.698604 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.698468 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d1e288a-8154-4e1e-bf7e-d7604b831916" containerName="istio-proxy" Jul 31 23:42:33.698693 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.698659 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d1e288a-8154-4e1e-bf7e-d7604b831916" containerName="istio-proxy" Jul 31 23:42:33.698693 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.698676 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d1e288a-8154-4e1e-bf7e-d7604b831916" containerName="istio-proxy" Jul 31 23:42:33.706895 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.706873 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.709657 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.709631 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-3d60f7-openshift-default-dockercfg-kshk4\"" Jul 31 23:42:33.713884 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.713859 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc"] Jul 31 23:42:33.726545 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.726492 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:33.770036 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.769995 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-envoy\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.770036 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.770040 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-token\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.770259 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.770091 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-workload-certs\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.770259 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.770114 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-data\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.770259 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.770191 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-credential-socket\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.770259 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.770238 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjcd2\" (UniqueName: \"kubernetes.io/projected/929c1f5e-6f48-4ef2-a167-370a6610ad8a-kube-api-access-bjcd2\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.770393 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.770314 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-podinfo\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.770393 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.770345 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-workload-socket\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.770393 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.770370 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istiod-ca-cert\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.871321 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.871272 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-podinfo\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.871321 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.871332 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-workload-socket\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.871642 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.871367 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istiod-ca-cert\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.871642 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.871400 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-envoy\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.871642 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.871437 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-token\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.871642 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.871601 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-workload-certs\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.871926 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.871651 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-data\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.871926 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.871721 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-credential-socket\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.871926 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.871775 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjcd2\" (UniqueName: \"kubernetes.io/projected/929c1f5e-6f48-4ef2-a167-370a6610ad8a-kube-api-access-bjcd2\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.871926 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.871869 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-workload-socket\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.872131 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.872008 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-data\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.872131 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.872080 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-workload-certs\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.872220 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.872131 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-credential-socket\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.872300 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.872278 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istiod-ca-cert\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.873781 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.873762 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-envoy\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.874006 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.873989 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-podinfo\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.879392 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.879363 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-token\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:33.879691 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:33.879671 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjcd2\" (UniqueName: \"kubernetes.io/projected/929c1f5e-6f48-4ef2-a167-370a6610ad8a-kube-api-access-bjcd2\") pod \"e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:34.020261 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:34.020162 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:34.163034 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:34.163007 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc"] Jul 31 23:42:34.165407 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:42:34.165380 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod929c1f5e_6f48_4ef2_a167_370a6610ad8a.slice/crio-e7d62aaa30b0220ec9bab8f2aac3fe9c6c23bc092fa0d1a7b3c23fcc01d20b1c WatchSource:0}: Error finding container e7d62aaa30b0220ec9bab8f2aac3fe9c6c23bc092fa0d1a7b3c23fcc01d20b1c: Status 404 returned error can't find the container with id e7d62aaa30b0220ec9bab8f2aac3fe9c6c23bc092fa0d1a7b3c23fcc01d20b1c Jul 31 23:42:34.167443 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:34.167403 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:34.167671 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:34.167643 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:42:34.167737 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:34.167711 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:42:34.167796 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:34.167738 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:42:34.477454 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:34.477415 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" event={"ID":"929c1f5e-6f48-4ef2-a167-370a6610ad8a","Type":"ContainerStarted","Data":"2fc353c692f2f4b969b6dfbc9d6a97e37156a888072324705ed8a73c3ead138b"} Jul 31 23:42:34.477679 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:34.477482 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" event={"ID":"929c1f5e-6f48-4ef2-a167-370a6610ad8a","Type":"ContainerStarted","Data":"e7d62aaa30b0220ec9bab8f2aac3fe9c6c23bc092fa0d1a7b3c23fcc01d20b1c"} Jul 31 23:42:34.520291 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:34.520233 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" podStartSLOduration=1.520213215 podStartE2EDuration="1.520213215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:42:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:42:34.517258467 +0000 UTC m=+2710.989702646" watchObservedRunningTime="2026-07-31 23:42:34.520213215 +0000 UTC m=+2710.992657398" Jul 31 23:42:35.021412 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:35.021369 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:35.026210 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:35.026184 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:35.485268 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:35.485231 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:35.486720 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:35.486696 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:36.472780 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:36.472744 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:36.572429 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:36.572383 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:37.606116 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:37.606082 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:38.074315 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:38.074277 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:38.180763 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:38.180734 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:38.401975 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:38.401895 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:44.241202 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:44.241145 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:42:50.551595 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:50.551367 2561 patch_prober.go:28] interesting pod/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": read tcp 10.134.0.2:51148->10.134.0.57:15021: read: connection reset by peer" start-of-body= Jul 31 23:42:50.551595 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:50.551445 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" podUID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": read tcp 10.134.0.2:51148->10.134.0.57:15021: read: connection reset by peer" Jul 31 23:42:51.575680 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:51.575647 2561 generic.go:356] "Generic (PLEG): container finished" podID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" containerID="2fc353c692f2f4b969b6dfbc9d6a97e37156a888072324705ed8a73c3ead138b" exitCode=0 Jul 31 23:42:51.576081 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:51.575723 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" event={"ID":"929c1f5e-6f48-4ef2-a167-370a6610ad8a","Type":"ContainerDied","Data":"2fc353c692f2f4b969b6dfbc9d6a97e37156a888072324705ed8a73c3ead138b"} Jul 31 23:42:51.576154 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:51.576140 2561 scope.go:117] "RemoveContainer" containerID="2fc353c692f2f4b969b6dfbc9d6a97e37156a888072324705ed8a73c3ead138b" Jul 31 23:42:51.577261 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:51.577234 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:42:51.577324 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:51.577304 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:42:51.577363 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:51.577335 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 31 23:42:52.584102 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:52.584065 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" event={"ID":"929c1f5e-6f48-4ef2-a167-370a6610ad8a","Type":"ContainerStarted","Data":"0946d049c38e4695bba2adb857c34371354f86a7df6e290883d3d6e84ea99698"} Jul 31 23:42:53.021352 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:53.021265 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:53.021574 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:53.021545 2561 patch_prober.go:28] interesting pod/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 31 23:42:53.021651 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:53.021607 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" podUID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 31 23:42:54.021305 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:54.021264 2561 patch_prober.go:28] interesting pod/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 31 23:42:54.021741 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:54.021331 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" podUID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 31 23:42:55.020710 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:55.020665 2561 patch_prober.go:28] interesting pod/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 31 23:42:55.020894 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:55.020736 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" podUID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 31 23:42:56.020784 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:56.020744 2561 patch_prober.go:28] interesting pod/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 31 23:42:56.021163 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:56.020810 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" podUID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 31 23:42:57.021343 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:57.021300 2561 patch_prober.go:28] interesting pod/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 31 23:42:57.021788 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:57.021364 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" podUID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 31 23:42:58.024827 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:58.024793 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:58.025222 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:58.025127 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:42:58.025761 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:42:58.025739 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:43:01.342488 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:43:01.342432 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:43:11.822258 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:43:11.822216 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:43:41.822451 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:43:41.822399 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:46:22.565295 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:46:22.565263 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:46:23.586425 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:46:23.586375 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:46:37.864013 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:46:37.863980 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:46:38.629682 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:46:38.629645 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:46:58.607919 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:46:58.607879 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:47:00.773988 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:47:00.773939 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:47:18.009552 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:47:18.009480 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:47:49.410101 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:47:49.410060 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:48:34.086371 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:48:34.086284 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:48:34.894857 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:48:34.894816 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:48:40.686870 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:48:40.686829 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:48:50.451714 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:48:50.451679 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:49:05.999385 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:49:05.999343 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:50:16.505031 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:16.504986 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:50:17.336005 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:17.335959 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:50:36.361015 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:36.360980 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:50:36.654424 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:36.654271 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:50:36.811925 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:36.811887 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:50:42.658672 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:42.658633 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:50:42.798825 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:42.798789 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:50:43.884964 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:43.884922 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:50:43.985874 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:43.985837 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:50:44.430111 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.430065 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:50:44.530670 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.530634 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:50:44.539441 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.539410 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx"] Jul 31 23:50:44.540097 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.540080 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d1e288a-8154-4e1e-bf7e-d7604b831916" containerName="istio-proxy" Jul 31 23:50:44.540145 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.540101 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d1e288a-8154-4e1e-bf7e-d7604b831916" containerName="istio-proxy" Jul 31 23:50:44.543676 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.543658 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" Jul 31 23:50:44.547488 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.547442 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-c16217\"" Jul 31 23:50:44.547712 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.547695 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-c16217-dockercfg-pqqks\"" Jul 31 23:50:44.556693 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.556670 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx"] Jul 31 23:50:44.570697 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.570661 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp"] Jul 31 23:50:44.574869 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.574831 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" Jul 31 23:50:44.584040 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.584016 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp"] Jul 31 23:50:44.594224 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.594194 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jppzq\" (UniqueName: \"kubernetes.io/projected/3f70c2a9-37f5-4df0-9978-dc3c766bd87b-kube-api-access-jppzq\") pod \"payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx\" (UID: \"3f70c2a9-37f5-4df0-9978-dc3c766bd87b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" Jul 31 23:50:44.594373 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.594350 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f70c2a9-37f5-4df0-9978-dc3c766bd87b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx\" (UID: \"3f70c2a9-37f5-4df0-9978-dc3c766bd87b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" Jul 31 23:50:44.695922 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.695827 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/452e15d0-d4e3-4cc0-87eb-26ed6c36f76c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp\" (UID: \"452e15d0-d4e3-4cc0-87eb-26ed6c36f76c\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" Jul 31 23:50:44.695922 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.695922 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f70c2a9-37f5-4df0-9978-dc3c766bd87b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx\" (UID: \"3f70c2a9-37f5-4df0-9978-dc3c766bd87b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" Jul 31 23:50:44.696122 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.695976 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jppzq\" (UniqueName: \"kubernetes.io/projected/3f70c2a9-37f5-4df0-9978-dc3c766bd87b-kube-api-access-jppzq\") pod \"payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx\" (UID: \"3f70c2a9-37f5-4df0-9978-dc3c766bd87b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" Jul 31 23:50:44.696122 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.696025 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v8qv\" (UniqueName: \"kubernetes.io/projected/452e15d0-d4e3-4cc0-87eb-26ed6c36f76c-kube-api-access-6v8qv\") pod \"payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp\" (UID: \"452e15d0-d4e3-4cc0-87eb-26ed6c36f76c\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" Jul 31 23:50:44.696723 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.696702 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f70c2a9-37f5-4df0-9978-dc3c766bd87b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx\" (UID: \"3f70c2a9-37f5-4df0-9978-dc3c766bd87b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" Jul 31 23:50:44.703903 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.703870 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jppzq\" (UniqueName: \"kubernetes.io/projected/3f70c2a9-37f5-4df0-9978-dc3c766bd87b-kube-api-access-jppzq\") pod \"payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx\" (UID: \"3f70c2a9-37f5-4df0-9978-dc3c766bd87b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" Jul 31 23:50:44.797651 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.797612 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v8qv\" (UniqueName: \"kubernetes.io/projected/452e15d0-d4e3-4cc0-87eb-26ed6c36f76c-kube-api-access-6v8qv\") pod \"payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp\" (UID: \"452e15d0-d4e3-4cc0-87eb-26ed6c36f76c\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" Jul 31 23:50:44.797829 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.797664 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/452e15d0-d4e3-4cc0-87eb-26ed6c36f76c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp\" (UID: \"452e15d0-d4e3-4cc0-87eb-26ed6c36f76c\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" Jul 31 23:50:44.798437 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.798416 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/452e15d0-d4e3-4cc0-87eb-26ed6c36f76c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp\" (UID: \"452e15d0-d4e3-4cc0-87eb-26ed6c36f76c\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" Jul 31 23:50:44.805056 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.805025 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v8qv\" (UniqueName: \"kubernetes.io/projected/452e15d0-d4e3-4cc0-87eb-26ed6c36f76c-kube-api-access-6v8qv\") pod \"payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp\" (UID: \"452e15d0-d4e3-4cc0-87eb-26ed6c36f76c\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" Jul 31 23:50:44.854658 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.854623 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" Jul 31 23:50:44.887308 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:44.887270 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" Jul 31 23:50:45.044423 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:45.044393 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp"] Jul 31 23:50:45.046702 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:50:45.046676 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod452e15d0_d4e3_4cc0_87eb_26ed6c36f76c.slice/crio-f35207b76367184d26a5d258ce3dd4f9503765650e7c05a5615b2ba7936c8ee3 WatchSource:0}: Error finding container f35207b76367184d26a5d258ce3dd4f9503765650e7c05a5615b2ba7936c8ee3: Status 404 returned error can't find the container with id f35207b76367184d26a5d258ce3dd4f9503765650e7c05a5615b2ba7936c8ee3 Jul 31 23:50:45.152175 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:45.152132 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" event={"ID":"452e15d0-d4e3-4cc0-87eb-26ed6c36f76c","Type":"ContainerStarted","Data":"68e8b2f700102e8161eee55f46a6985f8d462a8f3ec14af4b4aba90a4171cd9a"} Jul 31 23:50:45.152382 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:45.152205 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" Jul 31 23:50:45.152382 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:45.152230 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" event={"ID":"452e15d0-d4e3-4cc0-87eb-26ed6c36f76c","Type":"ContainerStarted","Data":"f35207b76367184d26a5d258ce3dd4f9503765650e7c05a5615b2ba7936c8ee3"} Jul 31 23:50:45.171126 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:45.171074 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" podStartSLOduration=1.171056474 podStartE2EDuration="1.171056474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:50:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:50:45.168680762 +0000 UTC m=+3201.641124966" watchObservedRunningTime="2026-07-31 23:50:45.171056474 +0000 UTC m=+3201.643500657" Jul 31 23:50:45.217483 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:45.217451 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx"] Jul 31 23:50:45.218839 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:50:45.218812 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f70c2a9_37f5_4df0_9978_dc3c766bd87b.slice/crio-33be97f2d5f16aeacd37e3f2cfec8c95098ea624d562c6ba14f70b460dc827e1 WatchSource:0}: Error finding container 33be97f2d5f16aeacd37e3f2cfec8c95098ea624d562c6ba14f70b460dc827e1: Status 404 returned error can't find the container with id 33be97f2d5f16aeacd37e3f2cfec8c95098ea624d562c6ba14f70b460dc827e1 Jul 31 23:50:46.159369 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:46.159324 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" event={"ID":"3f70c2a9-37f5-4df0-9978-dc3c766bd87b","Type":"ContainerStarted","Data":"54b9e437d1739c9384f00ac9eebb02881e5faa90129dae2c03cf362f2e8b684c"} Jul 31 23:50:46.159821 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:46.159612 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" Jul 31 23:50:46.159821 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:46.159661 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" event={"ID":"3f70c2a9-37f5-4df0-9978-dc3c766bd87b","Type":"ContainerStarted","Data":"33be97f2d5f16aeacd37e3f2cfec8c95098ea624d562c6ba14f70b460dc827e1"} Jul 31 23:50:46.181307 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:46.181256 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" podStartSLOduration=2.181240468 podStartE2EDuration="2.181240468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:50:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:50:46.178481304 +0000 UTC m=+3202.650925489" watchObservedRunningTime="2026-07-31 23:50:46.181240468 +0000 UTC m=+3202.653684651" Jul 31 23:50:56.161449 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:56.161418 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" Jul 31 23:50:57.168058 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:50:57.168025 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" Jul 31 23:51:07.633462 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:07.633427 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:51:08.291994 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:08.291950 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:51:08.665983 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:08.665941 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:51:09.314532 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:09.314472 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:51:11.849126 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:11.849085 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:51:12.002239 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:12.002200 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:51:12.975969 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:12.975935 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:51:13.218602 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:13.218565 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:51:13.594573 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:13.594542 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:51:13.699868 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:13.699834 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:51:36.730049 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:36.729957 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:51:58.109968 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.109927 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv"] Jul 31 23:51:58.113762 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.113733 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.116492 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.116463 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-c16217\"/\"openshift-service-ca.crt\"" Jul 31 23:51:58.116643 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.116594 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-c16217\"/\"kube-root-ca.crt\"" Jul 31 23:51:58.117615 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.117600 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-c16217\"/\"default-dockercfg-mmbgh\"" Jul 31 23:51:58.117762 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.117744 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-c16217\"/\"ipp-route-c16217-kserve-self-signed-certs\"" Jul 31 23:51:58.135549 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.135520 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv"] Jul 31 23:51:58.284272 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.284239 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-model-cache\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.284448 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.284284 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cea7a873-3f39-46d4-885d-d25f1774d288-tls-certs\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.284448 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.284315 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-tmp-dir\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.284448 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.284373 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grc4g\" (UniqueName: \"kubernetes.io/projected/cea7a873-3f39-46d4-885d-d25f1774d288-kube-api-access-grc4g\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.284448 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.284418 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-home\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.284617 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.284557 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-dshm\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.385384 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.385294 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-model-cache\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.385384 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.385341 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cea7a873-3f39-46d4-885d-d25f1774d288-tls-certs\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.385384 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.385366 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-tmp-dir\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.385681 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.385400 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grc4g\" (UniqueName: \"kubernetes.io/projected/cea7a873-3f39-46d4-885d-d25f1774d288-kube-api-access-grc4g\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.385681 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.385443 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-home\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.385681 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.385582 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-dshm\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.385804 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.385751 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-model-cache\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.385852 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.385834 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-home\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.385960 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.385943 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-tmp-dir\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.387753 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.387731 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-dshm\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.388058 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.388036 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cea7a873-3f39-46d4-885d-d25f1774d288-tls-certs\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.396710 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.396652 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grc4g\" (UniqueName: \"kubernetes.io/projected/cea7a873-3f39-46d4-885d-d25f1774d288-kube-api-access-grc4g\") pod \"ipp-route-c16217-kserve-7f9d9c4699-lfktv\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.423509 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.423477 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:51:58.568834 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:58.568733 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv"] Jul 31 23:51:58.571786 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:51:58.571761 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcea7a873_3f39_46d4_885d_d25f1774d288.slice/crio-b53afc5aaf249d6f033e07c0fef3963b97e0b6547fb7c63689990a048eaddcca WatchSource:0}: Error finding container b53afc5aaf249d6f033e07c0fef3963b97e0b6547fb7c63689990a048eaddcca: Status 404 returned error can't find the container with id b53afc5aaf249d6f033e07c0fef3963b97e0b6547fb7c63689990a048eaddcca Jul 31 23:51:59.382427 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:59.382375 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:51:59.568427 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:59.568392 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" event={"ID":"cea7a873-3f39-46d4-885d-d25f1774d288","Type":"ContainerStarted","Data":"0d51bc82a10f89b06e868ae96003486ce2c1ccef5f245b329e0af967c7b1cf7e"} Jul 31 23:51:59.568648 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:59.568437 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" event={"ID":"cea7a873-3f39-46d4-885d-d25f1774d288","Type":"ContainerStarted","Data":"b53afc5aaf249d6f033e07c0fef3963b97e0b6547fb7c63689990a048eaddcca"} Jul 31 23:51:59.588616 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:51:59.588565 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" podStartSLOduration=1.588549382 podStartE2EDuration="1.588549382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:51:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:51:59.586820975 +0000 UTC m=+3276.059265157" watchObservedRunningTime="2026-07-31 23:51:59.588549382 +0000 UTC m=+3276.060993586" Jul 31 23:52:08.424334 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:08.424287 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:52:08.424980 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:08.424347 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:52:08.436558 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:08.436525 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:52:08.628846 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:08.628810 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:52:38.458819 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:38.458778 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp_452e15d0-d4e3-4cc0-87eb-26ed6c36f76c/payload-processing/0.log" Jul 31 23:52:39.001731 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:39.001691 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:52:39.316177 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:39.316144 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:52:39.510890 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:39.510851 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:52:42.390367 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:42.390330 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:52:42.488037 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:42.487995 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:52:43.513440 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:43.513405 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:52:43.627986 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:43.627952 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:52:44.015287 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.015254 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:52:44.114642 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.114600 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:52:44.135509 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.135471 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw"] Jul 31 23:52:44.139269 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.139248 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" Jul 31 23:52:44.145839 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.145809 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-74ebf4-dockercfg-hvql9\"" Jul 31 23:52:44.145996 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.145810 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-74ebf4\"" Jul 31 23:52:44.146416 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.146393 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw"] Jul 31 23:52:44.172012 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.171981 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhvcv\" (UniqueName: \"kubernetes.io/projected/09b96fe4-4714-4c78-a3f2-0be0520fe0ed-kube-api-access-zhvcv\") pod \"payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw\" (UID: \"09b96fe4-4714-4c78-a3f2-0be0520fe0ed\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" Jul 31 23:52:44.172197 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.172051 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09b96fe4-4714-4c78-a3f2-0be0520fe0ed-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw\" (UID: \"09b96fe4-4714-4c78-a3f2-0be0520fe0ed\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" Jul 31 23:52:44.273661 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.273559 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhvcv\" (UniqueName: \"kubernetes.io/projected/09b96fe4-4714-4c78-a3f2-0be0520fe0ed-kube-api-access-zhvcv\") pod \"payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw\" (UID: \"09b96fe4-4714-4c78-a3f2-0be0520fe0ed\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" Jul 31 23:52:44.273837 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.273673 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09b96fe4-4714-4c78-a3f2-0be0520fe0ed-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw\" (UID: \"09b96fe4-4714-4c78-a3f2-0be0520fe0ed\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" Jul 31 23:52:44.274380 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.274359 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09b96fe4-4714-4c78-a3f2-0be0520fe0ed-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw\" (UID: \"09b96fe4-4714-4c78-a3f2-0be0520fe0ed\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" Jul 31 23:52:44.284108 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.284079 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhvcv\" (UniqueName: \"kubernetes.io/projected/09b96fe4-4714-4c78-a3f2-0be0520fe0ed-kube-api-access-zhvcv\") pod \"payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw\" (UID: \"09b96fe4-4714-4c78-a3f2-0be0520fe0ed\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" Jul 31 23:52:44.451905 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.451869 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" Jul 31 23:52:44.601154 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.601129 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw"] Jul 31 23:52:44.829219 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.829186 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" event={"ID":"09b96fe4-4714-4c78-a3f2-0be0520fe0ed","Type":"ContainerStarted","Data":"f57ef16dfe953bbcbcde4b5fc88ff46d370f83f4a4aa731f3ac7b73d7ce35d89"} Jul 31 23:52:44.829415 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.829235 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" event={"ID":"09b96fe4-4714-4c78-a3f2-0be0520fe0ed","Type":"ContainerStarted","Data":"5579432597f8356d29c00469857c23b7d6386f356ca357828e84dece57ec7603"} Jul 31 23:52:44.829415 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.829290 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" Jul 31 23:52:44.847157 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:44.847100 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" podStartSLOduration=0.847079495 podStartE2EDuration="847.079495ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:52:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:52:44.845092247 +0000 UTC m=+3321.317536433" watchObservedRunningTime="2026-07-31 23:52:44.847079495 +0000 UTC m=+3321.319523679" Jul 31 23:52:55.839283 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:52:55.839248 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" Jul 31 23:53:07.356613 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:07.356577 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:14.207335 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:14.207287 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:14.232084 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:14.232048 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw"] Jul 31 23:53:14.232349 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:14.232317 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" podUID="09b96fe4-4714-4c78-a3f2-0be0520fe0ed" containerName="payload-processing" containerID="cri-o://f57ef16dfe953bbcbcde4b5fc88ff46d370f83f4a4aa731f3ac7b73d7ce35d89" gracePeriod=30 Jul 31 23:53:14.308805 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:14.308770 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:14.369964 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:14.369927 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:14.502816 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:14.502789 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" Jul 31 23:53:14.604395 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:14.604351 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhvcv\" (UniqueName: \"kubernetes.io/projected/09b96fe4-4714-4c78-a3f2-0be0520fe0ed-kube-api-access-zhvcv\") pod \"09b96fe4-4714-4c78-a3f2-0be0520fe0ed\" (UID: \"09b96fe4-4714-4c78-a3f2-0be0520fe0ed\") " Jul 31 23:53:14.604601 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:14.604419 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09b96fe4-4714-4c78-a3f2-0be0520fe0ed-plugins-config-volume\") pod \"09b96fe4-4714-4c78-a3f2-0be0520fe0ed\" (UID: \"09b96fe4-4714-4c78-a3f2-0be0520fe0ed\") " Jul 31 23:53:14.604867 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:14.604841 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09b96fe4-4714-4c78-a3f2-0be0520fe0ed-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "09b96fe4-4714-4c78-a3f2-0be0520fe0ed" (UID: "09b96fe4-4714-4c78-a3f2-0be0520fe0ed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:53:14.606591 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:14.606569 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09b96fe4-4714-4c78-a3f2-0be0520fe0ed-kube-api-access-zhvcv" (OuterVolumeSpecName: "kube-api-access-zhvcv") pod "09b96fe4-4714-4c78-a3f2-0be0520fe0ed" (UID: "09b96fe4-4714-4c78-a3f2-0be0520fe0ed"). InnerVolumeSpecName "kube-api-access-zhvcv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:53:14.705244 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:14.705205 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhvcv\" (UniqueName: \"kubernetes.io/projected/09b96fe4-4714-4c78-a3f2-0be0520fe0ed-kube-api-access-zhvcv\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:53:14.705244 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:14.705235 2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09b96fe4-4714-4c78-a3f2-0be0520fe0ed-plugins-config-volume\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:53:14.711798 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:14.711756 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:15.002953 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:15.002917 2561 generic.go:356] "Generic (PLEG): container finished" podID="09b96fe4-4714-4c78-a3f2-0be0520fe0ed" containerID="f57ef16dfe953bbcbcde4b5fc88ff46d370f83f4a4aa731f3ac7b73d7ce35d89" exitCode=0 Jul 31 23:53:15.002953 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:15.002959 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" Jul 31 23:53:15.003227 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:15.003008 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" event={"ID":"09b96fe4-4714-4c78-a3f2-0be0520fe0ed","Type":"ContainerDied","Data":"f57ef16dfe953bbcbcde4b5fc88ff46d370f83f4a4aa731f3ac7b73d7ce35d89"} Jul 31 23:53:15.003227 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:15.003060 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw" event={"ID":"09b96fe4-4714-4c78-a3f2-0be0520fe0ed","Type":"ContainerDied","Data":"5579432597f8356d29c00469857c23b7d6386f356ca357828e84dece57ec7603"} Jul 31 23:53:15.003227 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:15.003082 2561 scope.go:117] "RemoveContainer" containerID="f57ef16dfe953bbcbcde4b5fc88ff46d370f83f4a4aa731f3ac7b73d7ce35d89" Jul 31 23:53:15.013300 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:15.013281 2561 scope.go:117] "RemoveContainer" containerID="f57ef16dfe953bbcbcde4b5fc88ff46d370f83f4a4aa731f3ac7b73d7ce35d89" Jul 31 23:53:15.013653 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:53:15.013626 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f57ef16dfe953bbcbcde4b5fc88ff46d370f83f4a4aa731f3ac7b73d7ce35d89\": container with ID starting with f57ef16dfe953bbcbcde4b5fc88ff46d370f83f4a4aa731f3ac7b73d7ce35d89 not found: ID does not exist" containerID="f57ef16dfe953bbcbcde4b5fc88ff46d370f83f4a4aa731f3ac7b73d7ce35d89" Jul 31 23:53:15.013718 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:15.013663 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f57ef16dfe953bbcbcde4b5fc88ff46d370f83f4a4aa731f3ac7b73d7ce35d89"} err="failed to get container status \"f57ef16dfe953bbcbcde4b5fc88ff46d370f83f4a4aa731f3ac7b73d7ce35d89\": rpc error: code = NotFound desc = could not find container \"f57ef16dfe953bbcbcde4b5fc88ff46d370f83f4a4aa731f3ac7b73d7ce35d89\": container with ID starting with f57ef16dfe953bbcbcde4b5fc88ff46d370f83f4a4aa731f3ac7b73d7ce35d89 not found: ID does not exist" Jul 31 23:53:15.026690 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:15.026661 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw"] Jul 31 23:53:15.032699 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:15.032673 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-74ebf4-8466894cb8-bmzbw"] Jul 31 23:53:16.133599 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:16.133563 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09b96fe4-4714-4c78-a3f2-0be0520fe0ed" path="/var/lib/kubelet/pods/09b96fe4-4714-4c78-a3f2-0be0520fe0ed/volumes" Jul 31 23:53:20.497281 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:20.497245 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:20.599028 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:20.598984 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:20.755435 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:20.755335 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:20.973449 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:20.973409 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:39.746707 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:39.746663 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:39.778682 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:39.778254 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp"] Jul 31 23:53:39.778682 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:39.778583 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" podUID="452e15d0-d4e3-4cc0-87eb-26ed6c36f76c" containerName="payload-processing" containerID="cri-o://68e8b2f700102e8161eee55f46a6985f8d462a8f3ec14af4b4aba90a4171cd9a" gracePeriod=30 Jul 31 23:53:39.781621 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:39.781592 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx"] Jul 31 23:53:39.782025 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:39.781994 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" podUID="3f70c2a9-37f5-4df0-9978-dc3c766bd87b" containerName="payload-pre-processing" containerID="cri-o://54b9e437d1739c9384f00ac9eebb02881e5faa90129dae2c03cf362f2e8b684c" gracePeriod=30 Jul 31 23:53:39.821976 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:39.821924 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:39.881052 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:39.881012 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:39.954990 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:39.954454 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:40.085598 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.085563 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" Jul 31 23:53:40.088794 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.088769 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" Jul 31 23:53:40.144646 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.144613 2561 generic.go:356] "Generic (PLEG): container finished" podID="452e15d0-d4e3-4cc0-87eb-26ed6c36f76c" containerID="68e8b2f700102e8161eee55f46a6985f8d462a8f3ec14af4b4aba90a4171cd9a" exitCode=0 Jul 31 23:53:40.144836 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.144657 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" Jul 31 23:53:40.144836 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.144697 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" event={"ID":"452e15d0-d4e3-4cc0-87eb-26ed6c36f76c","Type":"ContainerDied","Data":"68e8b2f700102e8161eee55f46a6985f8d462a8f3ec14af4b4aba90a4171cd9a"} Jul 31 23:53:40.144836 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.144764 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp" event={"ID":"452e15d0-d4e3-4cc0-87eb-26ed6c36f76c","Type":"ContainerDied","Data":"f35207b76367184d26a5d258ce3dd4f9503765650e7c05a5615b2ba7936c8ee3"} Jul 31 23:53:40.144836 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.144796 2561 scope.go:117] "RemoveContainer" containerID="68e8b2f700102e8161eee55f46a6985f8d462a8f3ec14af4b4aba90a4171cd9a" Jul 31 23:53:40.146043 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.146020 2561 generic.go:356] "Generic (PLEG): container finished" podID="3f70c2a9-37f5-4df0-9978-dc3c766bd87b" containerID="54b9e437d1739c9384f00ac9eebb02881e5faa90129dae2c03cf362f2e8b684c" exitCode=0 Jul 31 23:53:40.146163 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.146066 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" Jul 31 23:53:40.146163 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.146101 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" event={"ID":"3f70c2a9-37f5-4df0-9978-dc3c766bd87b","Type":"ContainerDied","Data":"54b9e437d1739c9384f00ac9eebb02881e5faa90129dae2c03cf362f2e8b684c"} Jul 31 23:53:40.146163 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.146150 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx" event={"ID":"3f70c2a9-37f5-4df0-9978-dc3c766bd87b","Type":"ContainerDied","Data":"33be97f2d5f16aeacd37e3f2cfec8c95098ea624d562c6ba14f70b460dc827e1"} Jul 31 23:53:40.154488 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.154461 2561 scope.go:117] "RemoveContainer" containerID="68e8b2f700102e8161eee55f46a6985f8d462a8f3ec14af4b4aba90a4171cd9a" Jul 31 23:53:40.154810 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:53:40.154784 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68e8b2f700102e8161eee55f46a6985f8d462a8f3ec14af4b4aba90a4171cd9a\": container with ID starting with 68e8b2f700102e8161eee55f46a6985f8d462a8f3ec14af4b4aba90a4171cd9a not found: ID does not exist" containerID="68e8b2f700102e8161eee55f46a6985f8d462a8f3ec14af4b4aba90a4171cd9a" Jul 31 23:53:40.154870 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.154819 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68e8b2f700102e8161eee55f46a6985f8d462a8f3ec14af4b4aba90a4171cd9a"} err="failed to get container status \"68e8b2f700102e8161eee55f46a6985f8d462a8f3ec14af4b4aba90a4171cd9a\": rpc error: code = NotFound desc = could not find container \"68e8b2f700102e8161eee55f46a6985f8d462a8f3ec14af4b4aba90a4171cd9a\": container with ID starting with 68e8b2f700102e8161eee55f46a6985f8d462a8f3ec14af4b4aba90a4171cd9a not found: ID does not exist" Jul 31 23:53:40.154870 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.154835 2561 scope.go:117] "RemoveContainer" containerID="54b9e437d1739c9384f00ac9eebb02881e5faa90129dae2c03cf362f2e8b684c" Jul 31 23:53:40.163026 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.163006 2561 scope.go:117] "RemoveContainer" containerID="54b9e437d1739c9384f00ac9eebb02881e5faa90129dae2c03cf362f2e8b684c" Jul 31 23:53:40.163288 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:53:40.163269 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54b9e437d1739c9384f00ac9eebb02881e5faa90129dae2c03cf362f2e8b684c\": container with ID starting with 54b9e437d1739c9384f00ac9eebb02881e5faa90129dae2c03cf362f2e8b684c not found: ID does not exist" containerID="54b9e437d1739c9384f00ac9eebb02881e5faa90129dae2c03cf362f2e8b684c" Jul 31 23:53:40.163352 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.163298 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54b9e437d1739c9384f00ac9eebb02881e5faa90129dae2c03cf362f2e8b684c"} err="failed to get container status \"54b9e437d1739c9384f00ac9eebb02881e5faa90129dae2c03cf362f2e8b684c\": rpc error: code = NotFound desc = could not find container \"54b9e437d1739c9384f00ac9eebb02881e5faa90129dae2c03cf362f2e8b684c\": container with ID starting with 54b9e437d1739c9384f00ac9eebb02881e5faa90129dae2c03cf362f2e8b684c not found: ID does not exist" Jul 31 23:53:40.178753 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.178720 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f70c2a9-37f5-4df0-9978-dc3c766bd87b-plugins-config-volume\") pod \"3f70c2a9-37f5-4df0-9978-dc3c766bd87b\" (UID: \"3f70c2a9-37f5-4df0-9978-dc3c766bd87b\") " Jul 31 23:53:40.178959 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.178772 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/452e15d0-d4e3-4cc0-87eb-26ed6c36f76c-plugins-config-volume\") pod \"452e15d0-d4e3-4cc0-87eb-26ed6c36f76c\" (UID: \"452e15d0-d4e3-4cc0-87eb-26ed6c36f76c\") " Jul 31 23:53:40.178959 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.178807 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jppzq\" (UniqueName: \"kubernetes.io/projected/3f70c2a9-37f5-4df0-9978-dc3c766bd87b-kube-api-access-jppzq\") pod \"3f70c2a9-37f5-4df0-9978-dc3c766bd87b\" (UID: \"3f70c2a9-37f5-4df0-9978-dc3c766bd87b\") " Jul 31 23:53:40.178959 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.178878 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v8qv\" (UniqueName: \"kubernetes.io/projected/452e15d0-d4e3-4cc0-87eb-26ed6c36f76c-kube-api-access-6v8qv\") pod \"452e15d0-d4e3-4cc0-87eb-26ed6c36f76c\" (UID: \"452e15d0-d4e3-4cc0-87eb-26ed6c36f76c\") " Jul 31 23:53:40.179208 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.179183 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f70c2a9-37f5-4df0-9978-dc3c766bd87b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3f70c2a9-37f5-4df0-9978-dc3c766bd87b" (UID: "3f70c2a9-37f5-4df0-9978-dc3c766bd87b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:53:40.179284 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.179218 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/452e15d0-d4e3-4cc0-87eb-26ed6c36f76c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "452e15d0-d4e3-4cc0-87eb-26ed6c36f76c" (UID: "452e15d0-d4e3-4cc0-87eb-26ed6c36f76c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:53:40.180958 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.180932 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f70c2a9-37f5-4df0-9978-dc3c766bd87b-kube-api-access-jppzq" (OuterVolumeSpecName: "kube-api-access-jppzq") pod "3f70c2a9-37f5-4df0-9978-dc3c766bd87b" (UID: "3f70c2a9-37f5-4df0-9978-dc3c766bd87b"). InnerVolumeSpecName "kube-api-access-jppzq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:53:40.181053 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.181030 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/452e15d0-d4e3-4cc0-87eb-26ed6c36f76c-kube-api-access-6v8qv" (OuterVolumeSpecName: "kube-api-access-6v8qv") pod "452e15d0-d4e3-4cc0-87eb-26ed6c36f76c" (UID: "452e15d0-d4e3-4cc0-87eb-26ed6c36f76c"). InnerVolumeSpecName "kube-api-access-6v8qv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:53:40.280560 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.280437 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v8qv\" (UniqueName: \"kubernetes.io/projected/452e15d0-d4e3-4cc0-87eb-26ed6c36f76c-kube-api-access-6v8qv\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:53:40.280560 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.280472 2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f70c2a9-37f5-4df0-9978-dc3c766bd87b-plugins-config-volume\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:53:40.280560 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.280481 2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/452e15d0-d4e3-4cc0-87eb-26ed6c36f76c-plugins-config-volume\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:53:40.280560 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.280490 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jppzq\" (UniqueName: \"kubernetes.io/projected/3f70c2a9-37f5-4df0-9978-dc3c766bd87b-kube-api-access-jppzq\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:53:40.282840 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.282802 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:40.469916 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.469882 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp"] Jul 31 23:53:40.474716 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.474689 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c16217-68f8b7b58-gfvcp"] Jul 31 23:53:40.485420 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.485393 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx"] Jul 31 23:53:40.488979 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:40.488957 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c16217-5b74b95d4-jjxjx"] Jul 31 23:53:42.135604 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:42.135556 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f70c2a9-37f5-4df0-9978-dc3c766bd87b" path="/var/lib/kubelet/pods/3f70c2a9-37f5-4df0-9978-dc3c766bd87b/volumes" Jul 31 23:53:42.136093 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:42.136076 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="452e15d0-d4e3-4cc0-87eb-26ed6c36f76c" path="/var/lib/kubelet/pods/452e15d0-d4e3-4cc0-87eb-26ed6c36f76c/volumes" Jul 31 23:53:44.964744 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:44.964700 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:45.269160 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:45.269069 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv"] Jul 31 23:53:45.363753 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:45.363711 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:58.997871 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:58.997831 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:53:59.050551 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:53:59.050491 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:00.289475 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.289430 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" podUID="cea7a873-3f39-46d4-885d-d25f1774d288" containerName="main" containerID="cri-o://0d51bc82a10f89b06e868ae96003486ce2c1ccef5f245b329e0af967c7b1cf7e" gracePeriod=45 Jul 31 23:54:00.534546 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.534520 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:54:00.696956 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.696915 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cea7a873-3f39-46d4-885d-d25f1774d288-tls-certs\") pod \"cea7a873-3f39-46d4-885d-d25f1774d288\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " Jul 31 23:54:00.697148 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.696966 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grc4g\" (UniqueName: \"kubernetes.io/projected/cea7a873-3f39-46d4-885d-d25f1774d288-kube-api-access-grc4g\") pod \"cea7a873-3f39-46d4-885d-d25f1774d288\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " Jul 31 23:54:00.697148 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.697015 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-dshm\") pod \"cea7a873-3f39-46d4-885d-d25f1774d288\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " Jul 31 23:54:00.697148 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.697058 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-tmp-dir\") pod \"cea7a873-3f39-46d4-885d-d25f1774d288\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " Jul 31 23:54:00.697148 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.697108 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-home\") pod \"cea7a873-3f39-46d4-885d-d25f1774d288\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " Jul 31 23:54:00.697148 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.697137 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-model-cache\") pod \"cea7a873-3f39-46d4-885d-d25f1774d288\" (UID: \"cea7a873-3f39-46d4-885d-d25f1774d288\") " Jul 31 23:54:00.697412 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.697348 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-home" (OuterVolumeSpecName: "home") pod "cea7a873-3f39-46d4-885d-d25f1774d288" (UID: "cea7a873-3f39-46d4-885d-d25f1774d288"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:54:00.697412 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.697364 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-model-cache" (OuterVolumeSpecName: "model-cache") pod "cea7a873-3f39-46d4-885d-d25f1774d288" (UID: "cea7a873-3f39-46d4-885d-d25f1774d288"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:54:00.697412 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.697374 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cea7a873-3f39-46d4-885d-d25f1774d288" (UID: "cea7a873-3f39-46d4-885d-d25f1774d288"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:54:00.697658 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.697623 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-tmp-dir\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:54:00.697720 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.697662 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-home\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:54:00.697720 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.697675 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-model-cache\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:54:00.699378 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.699351 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cea7a873-3f39-46d4-885d-d25f1774d288-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cea7a873-3f39-46d4-885d-d25f1774d288" (UID: "cea7a873-3f39-46d4-885d-d25f1774d288"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 23:54:00.699378 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.699361 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-dshm" (OuterVolumeSpecName: "dshm") pod "cea7a873-3f39-46d4-885d-d25f1774d288" (UID: "cea7a873-3f39-46d4-885d-d25f1774d288"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:54:00.699572 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.699418 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cea7a873-3f39-46d4-885d-d25f1774d288-kube-api-access-grc4g" (OuterVolumeSpecName: "kube-api-access-grc4g") pod "cea7a873-3f39-46d4-885d-d25f1774d288" (UID: "cea7a873-3f39-46d4-885d-d25f1774d288"). InnerVolumeSpecName "kube-api-access-grc4g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:54:00.798996 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.798955 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cea7a873-3f39-46d4-885d-d25f1774d288-tls-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:54:00.798996 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.798989 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grc4g\" (UniqueName: \"kubernetes.io/projected/cea7a873-3f39-46d4-885d-d25f1774d288-kube-api-access-grc4g\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:54:00.798996 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:00.799000 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cea7a873-3f39-46d4-885d-d25f1774d288-dshm\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:54:01.263031 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:01.262995 2561 generic.go:356] "Generic (PLEG): container finished" podID="cea7a873-3f39-46d4-885d-d25f1774d288" containerID="0d51bc82a10f89b06e868ae96003486ce2c1ccef5f245b329e0af967c7b1cf7e" exitCode=0 Jul 31 23:54:01.263207 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:01.263037 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" Jul 31 23:54:01.263207 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:01.263069 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" event={"ID":"cea7a873-3f39-46d4-885d-d25f1774d288","Type":"ContainerDied","Data":"0d51bc82a10f89b06e868ae96003486ce2c1ccef5f245b329e0af967c7b1cf7e"} Jul 31 23:54:01.263207 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:01.263106 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv" event={"ID":"cea7a873-3f39-46d4-885d-d25f1774d288","Type":"ContainerDied","Data":"b53afc5aaf249d6f033e07c0fef3963b97e0b6547fb7c63689990a048eaddcca"} Jul 31 23:54:01.263207 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:01.263124 2561 scope.go:117] "RemoveContainer" containerID="0d51bc82a10f89b06e868ae96003486ce2c1ccef5f245b329e0af967c7b1cf7e" Jul 31 23:54:01.272681 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:01.272661 2561 scope.go:117] "RemoveContainer" containerID="0d51bc82a10f89b06e868ae96003486ce2c1ccef5f245b329e0af967c7b1cf7e" Jul 31 23:54:01.272983 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:54:01.272964 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d51bc82a10f89b06e868ae96003486ce2c1ccef5f245b329e0af967c7b1cf7e\": container with ID starting with 0d51bc82a10f89b06e868ae96003486ce2c1ccef5f245b329e0af967c7b1cf7e not found: ID does not exist" containerID="0d51bc82a10f89b06e868ae96003486ce2c1ccef5f245b329e0af967c7b1cf7e" Jul 31 23:54:01.273054 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:01.272999 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d51bc82a10f89b06e868ae96003486ce2c1ccef5f245b329e0af967c7b1cf7e"} err="failed to get container status \"0d51bc82a10f89b06e868ae96003486ce2c1ccef5f245b329e0af967c7b1cf7e\": rpc error: code = NotFound desc = could not find container \"0d51bc82a10f89b06e868ae96003486ce2c1ccef5f245b329e0af967c7b1cf7e\": container with ID starting with 0d51bc82a10f89b06e868ae96003486ce2c1ccef5f245b329e0af967c7b1cf7e not found: ID does not exist" Jul 31 23:54:01.284747 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:01.284710 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv"] Jul 31 23:54:01.288391 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:01.288359 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-c16217/ipp-route-c16217-kserve-7f9d9c4699-lfktv"] Jul 31 23:54:02.133653 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:02.133619 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cea7a873-3f39-46d4-885d-d25f1774d288" path="/var/lib/kubelet/pods/cea7a873-3f39-46d4-885d-d25f1774d288/volumes" Jul 31 23:54:21.840462 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:21.840369 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:22.365104 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:22.365068 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:22.811423 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:22.811373 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:27.986155 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:27.986111 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:28.086495 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:28.086449 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:28.268625 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:28.268520 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:28.476562 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:28.476526 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:39.133144 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:39.133100 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:39.198921 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:39.198873 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:39.276736 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:39.276700 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:39.339258 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:39.339220 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:39.685593 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:39.685560 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:45.334496 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:45.334457 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:45.396386 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:45.396348 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:45.593709 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:45.593615 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:45.797312 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:45.797270 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:56.491528 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:56.491460 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:56.591385 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:56.591352 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:56.659917 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:56.659881 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:57.004143 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:57.004097 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:54:58.345281 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:54:58.345243 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:55:02.862927 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:02.862864 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:55:02.962959 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:02.962920 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:55:03.118060 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:03.117955 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck"] Jul 31 23:55:03.118248 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:03.118210 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" podUID="751e6f22-f48a-476f-bbea-570bd79d0d2e" containerName="istio-proxy" containerID="cri-o://c357b07ab2f7e9e4f433f9123c98f7bbc875c4525cf65d43593a43a471385c60" gracePeriod=30 Jul 31 23:55:03.138748 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:03.138706 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:55:03.345629 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:03.345584 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:55:08.386184 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.386154 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:55:08.471177 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471069 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-token\") pod \"751e6f22-f48a-476f-bbea-570bd79d0d2e\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " Jul 31 23:55:08.471177 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471115 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-data\") pod \"751e6f22-f48a-476f-bbea-570bd79d0d2e\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " Jul 31 23:55:08.471177 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471135 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-credential-socket\") pod \"751e6f22-f48a-476f-bbea-570bd79d0d2e\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " Jul 31 23:55:08.471494 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471221 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/751e6f22-f48a-476f-bbea-570bd79d0d2e-istiod-ca-cert\") pod \"751e6f22-f48a-476f-bbea-570bd79d0d2e\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " Jul 31 23:55:08.471494 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471276 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-podinfo\") pod \"751e6f22-f48a-476f-bbea-570bd79d0d2e\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " Jul 31 23:55:08.471494 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471304 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-workload-socket\") pod \"751e6f22-f48a-476f-bbea-570bd79d0d2e\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " Jul 31 23:55:08.471494 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471321 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-workload-certs\") pod \"751e6f22-f48a-476f-bbea-570bd79d0d2e\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " Jul 31 23:55:08.471494 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471358 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cz6g2\" (UniqueName: \"kubernetes.io/projected/751e6f22-f48a-476f-bbea-570bd79d0d2e-kube-api-access-cz6g2\") pod \"751e6f22-f48a-476f-bbea-570bd79d0d2e\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " Jul 31 23:55:08.471494 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471387 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-envoy\") pod \"751e6f22-f48a-476f-bbea-570bd79d0d2e\" (UID: \"751e6f22-f48a-476f-bbea-570bd79d0d2e\") " Jul 31 23:55:08.471494 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471406 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-data" (OuterVolumeSpecName: "istio-data") pod "751e6f22-f48a-476f-bbea-570bd79d0d2e" (UID: "751e6f22-f48a-476f-bbea-570bd79d0d2e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:55:08.471844 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471596 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "751e6f22-f48a-476f-bbea-570bd79d0d2e" (UID: "751e6f22-f48a-476f-bbea-570bd79d0d2e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:55:08.471844 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471660 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "751e6f22-f48a-476f-bbea-570bd79d0d2e" (UID: "751e6f22-f48a-476f-bbea-570bd79d0d2e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:55:08.471844 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471715 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/751e6f22-f48a-476f-bbea-570bd79d0d2e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "751e6f22-f48a-476f-bbea-570bd79d0d2e" (UID: "751e6f22-f48a-476f-bbea-570bd79d0d2e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:55:08.472012 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471841 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/751e6f22-f48a-476f-bbea-570bd79d0d2e-istiod-ca-cert\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:08.472012 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471859 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-workload-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:08.472012 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471872 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-data\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:08.472012 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471887 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-credential-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:08.472012 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.471909 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "751e6f22-f48a-476f-bbea-570bd79d0d2e" (UID: "751e6f22-f48a-476f-bbea-570bd79d0d2e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:55:08.473444 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.473419 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-token" (OuterVolumeSpecName: "istio-token") pod "751e6f22-f48a-476f-bbea-570bd79d0d2e" (UID: "751e6f22-f48a-476f-bbea-570bd79d0d2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:55:08.473561 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.473535 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "751e6f22-f48a-476f-bbea-570bd79d0d2e" (UID: "751e6f22-f48a-476f-bbea-570bd79d0d2e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:55:08.473561 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.473532 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "751e6f22-f48a-476f-bbea-570bd79d0d2e" (UID: "751e6f22-f48a-476f-bbea-570bd79d0d2e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 23:55:08.473648 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.473598 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/751e6f22-f48a-476f-bbea-570bd79d0d2e-kube-api-access-cz6g2" (OuterVolumeSpecName: "kube-api-access-cz6g2") pod "751e6f22-f48a-476f-bbea-570bd79d0d2e" (UID: "751e6f22-f48a-476f-bbea-570bd79d0d2e"). InnerVolumeSpecName "kube-api-access-cz6g2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:55:08.573266 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.573223 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-podinfo\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:08.573266 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.573258 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-workload-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:08.573266 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.573268 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cz6g2\" (UniqueName: \"kubernetes.io/projected/751e6f22-f48a-476f-bbea-570bd79d0d2e-kube-api-access-cz6g2\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:08.573266 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.573278 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-envoy\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:08.573586 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.573288 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/751e6f22-f48a-476f-bbea-570bd79d0d2e-istio-token\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:08.629249 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.629215 2561 generic.go:356] "Generic (PLEG): container finished" podID="751e6f22-f48a-476f-bbea-570bd79d0d2e" containerID="c357b07ab2f7e9e4f433f9123c98f7bbc875c4525cf65d43593a43a471385c60" exitCode=0 Jul 31 23:55:08.629457 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.629268 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" Jul 31 23:55:08.629457 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.629304 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" event={"ID":"751e6f22-f48a-476f-bbea-570bd79d0d2e","Type":"ContainerDied","Data":"c357b07ab2f7e9e4f433f9123c98f7bbc875c4525cf65d43593a43a471385c60"} Jul 31 23:55:08.629457 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.629361 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck" event={"ID":"751e6f22-f48a-476f-bbea-570bd79d0d2e","Type":"ContainerDied","Data":"409be6aa897dfaeaf8a748789e1d294e0145600fa87913c0bc805a493bf058fd"} Jul 31 23:55:08.629457 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.629384 2561 scope.go:117] "RemoveContainer" containerID="c357b07ab2f7e9e4f433f9123c98f7bbc875c4525cf65d43593a43a471385c60" Jul 31 23:55:08.639493 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.639472 2561 scope.go:117] "RemoveContainer" containerID="776083e0411b4b379981bad9264a31ff99728057ffb5a9d75edb58e46c5b88c9" Jul 31 23:55:08.648386 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.648365 2561 scope.go:117] "RemoveContainer" containerID="c357b07ab2f7e9e4f433f9123c98f7bbc875c4525cf65d43593a43a471385c60" Jul 31 23:55:08.648758 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:55:08.648737 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c357b07ab2f7e9e4f433f9123c98f7bbc875c4525cf65d43593a43a471385c60\": container with ID starting with c357b07ab2f7e9e4f433f9123c98f7bbc875c4525cf65d43593a43a471385c60 not found: ID does not exist" containerID="c357b07ab2f7e9e4f433f9123c98f7bbc875c4525cf65d43593a43a471385c60" Jul 31 23:55:08.648865 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.648765 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c357b07ab2f7e9e4f433f9123c98f7bbc875c4525cf65d43593a43a471385c60"} err="failed to get container status \"c357b07ab2f7e9e4f433f9123c98f7bbc875c4525cf65d43593a43a471385c60\": rpc error: code = NotFound desc = could not find container \"c357b07ab2f7e9e4f433f9123c98f7bbc875c4525cf65d43593a43a471385c60\": container with ID starting with c357b07ab2f7e9e4f433f9123c98f7bbc875c4525cf65d43593a43a471385c60 not found: ID does not exist" Jul 31 23:55:08.648865 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.648790 2561 scope.go:117] "RemoveContainer" containerID="776083e0411b4b379981bad9264a31ff99728057ffb5a9d75edb58e46c5b88c9" Jul 31 23:55:08.649744 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:55:08.649712 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"776083e0411b4b379981bad9264a31ff99728057ffb5a9d75edb58e46c5b88c9\": container with ID starting with 776083e0411b4b379981bad9264a31ff99728057ffb5a9d75edb58e46c5b88c9 not found: ID does not exist" containerID="776083e0411b4b379981bad9264a31ff99728057ffb5a9d75edb58e46c5b88c9" Jul 31 23:55:08.649889 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.649750 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"776083e0411b4b379981bad9264a31ff99728057ffb5a9d75edb58e46c5b88c9"} err="failed to get container status \"776083e0411b4b379981bad9264a31ff99728057ffb5a9d75edb58e46c5b88c9\": rpc error: code = NotFound desc = could not find container \"776083e0411b4b379981bad9264a31ff99728057ffb5a9d75edb58e46c5b88c9\": container with ID starting with 776083e0411b4b379981bad9264a31ff99728057ffb5a9d75edb58e46c5b88c9 not found: ID does not exist" Jul 31 23:55:08.651959 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.651941 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck"] Jul 31 23:55:08.657883 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:08.657859 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-99d0e4-openshift-default-79f44dbd76-v8tck"] Jul 31 23:55:10.133469 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:10.133430 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="751e6f22-f48a-476f-bbea-570bd79d0d2e" path="/var/lib/kubelet/pods/751e6f22-f48a-476f-bbea-570bd79d0d2e/volumes" Jul 31 23:55:13.860042 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:13.860001 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:55:13.960336 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:13.960297 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:55:14.018689 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:14.018648 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:55:14.417529 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:14.417471 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:55:20.221674 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:20.221634 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:55:20.288460 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:20.288415 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:55:20.439308 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:20.439275 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc"] Jul 31 23:55:20.439874 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:20.439840 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" podUID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" containerName="istio-proxy" containerID="cri-o://0946d049c38e4695bba2adb857c34371354f86a7df6e290883d3d6e84ea99698" gracePeriod=30 Jul 31 23:55:20.460105 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:20.460062 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:55:20.679251 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:20.679215 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:55:25.704186 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.704158 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:55:25.722196 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.722110 2561 generic.go:356] "Generic (PLEG): container finished" podID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" containerID="0946d049c38e4695bba2adb857c34371354f86a7df6e290883d3d6e84ea99698" exitCode=0 Jul 31 23:55:25.722196 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.722162 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" Jul 31 23:55:25.722196 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.722187 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" event={"ID":"929c1f5e-6f48-4ef2-a167-370a6610ad8a","Type":"ContainerDied","Data":"0946d049c38e4695bba2adb857c34371354f86a7df6e290883d3d6e84ea99698"} Jul 31 23:55:25.722439 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.722243 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc" event={"ID":"929c1f5e-6f48-4ef2-a167-370a6610ad8a","Type":"ContainerDied","Data":"e7d62aaa30b0220ec9bab8f2aac3fe9c6c23bc092fa0d1a7b3c23fcc01d20b1c"} Jul 31 23:55:25.722439 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.722265 2561 scope.go:117] "RemoveContainer" containerID="0946d049c38e4695bba2adb857c34371354f86a7df6e290883d3d6e84ea99698" Jul 31 23:55:25.733005 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.732979 2561 scope.go:117] "RemoveContainer" containerID="2fc353c692f2f4b969b6dfbc9d6a97e37156a888072324705ed8a73c3ead138b" Jul 31 23:55:25.742050 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.742029 2561 scope.go:117] "RemoveContainer" containerID="0946d049c38e4695bba2adb857c34371354f86a7df6e290883d3d6e84ea99698" Jul 31 23:55:25.742375 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:55:25.742354 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0946d049c38e4695bba2adb857c34371354f86a7df6e290883d3d6e84ea99698\": container with ID starting with 0946d049c38e4695bba2adb857c34371354f86a7df6e290883d3d6e84ea99698 not found: ID does not exist" containerID="0946d049c38e4695bba2adb857c34371354f86a7df6e290883d3d6e84ea99698" Jul 31 23:55:25.742434 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.742385 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0946d049c38e4695bba2adb857c34371354f86a7df6e290883d3d6e84ea99698"} err="failed to get container status \"0946d049c38e4695bba2adb857c34371354f86a7df6e290883d3d6e84ea99698\": rpc error: code = NotFound desc = could not find container \"0946d049c38e4695bba2adb857c34371354f86a7df6e290883d3d6e84ea99698\": container with ID starting with 0946d049c38e4695bba2adb857c34371354f86a7df6e290883d3d6e84ea99698 not found: ID does not exist" Jul 31 23:55:25.742434 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.742405 2561 scope.go:117] "RemoveContainer" containerID="2fc353c692f2f4b969b6dfbc9d6a97e37156a888072324705ed8a73c3ead138b" Jul 31 23:55:25.742720 ip-10-0-131-148 kubenswrapper[2561]: E0731 23:55:25.742701 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fc353c692f2f4b969b6dfbc9d6a97e37156a888072324705ed8a73c3ead138b\": container with ID starting with 2fc353c692f2f4b969b6dfbc9d6a97e37156a888072324705ed8a73c3ead138b not found: ID does not exist" containerID="2fc353c692f2f4b969b6dfbc9d6a97e37156a888072324705ed8a73c3ead138b" Jul 31 23:55:25.742800 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.742729 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fc353c692f2f4b969b6dfbc9d6a97e37156a888072324705ed8a73c3ead138b"} err="failed to get container status \"2fc353c692f2f4b969b6dfbc9d6a97e37156a888072324705ed8a73c3ead138b\": rpc error: code = NotFound desc = could not find container \"2fc353c692f2f4b969b6dfbc9d6a97e37156a888072324705ed8a73c3ead138b\": container with ID starting with 2fc353c692f2f4b969b6dfbc9d6a97e37156a888072324705ed8a73c3ead138b not found: ID does not exist" Jul 31 23:55:25.855269 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.855224 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-credential-socket\") pod \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " Jul 31 23:55:25.855481 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.855323 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istiod-ca-cert\") pod \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " Jul 31 23:55:25.855481 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.855388 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-workload-certs\") pod \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " Jul 31 23:55:25.855481 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.855444 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-envoy\") pod \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " Jul 31 23:55:25.855481 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.855468 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-data\") pod \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " Jul 31 23:55:25.855751 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.855554 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-podinfo\") pod \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " Jul 31 23:55:25.855751 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.855552 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "929c1f5e-6f48-4ef2-a167-370a6610ad8a" (UID: "929c1f5e-6f48-4ef2-a167-370a6610ad8a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:55:25.855751 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.855607 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-token\") pod \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " Jul 31 23:55:25.855751 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.855640 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjcd2\" (UniqueName: \"kubernetes.io/projected/929c1f5e-6f48-4ef2-a167-370a6610ad8a-kube-api-access-bjcd2\") pod \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " Jul 31 23:55:25.855751 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.855664 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-workload-socket\") pod \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\" (UID: \"929c1f5e-6f48-4ef2-a167-370a6610ad8a\") " Jul 31 23:55:25.855986 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.855809 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "929c1f5e-6f48-4ef2-a167-370a6610ad8a" (UID: "929c1f5e-6f48-4ef2-a167-370a6610ad8a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:55:25.855986 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.855818 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "929c1f5e-6f48-4ef2-a167-370a6610ad8a" (UID: "929c1f5e-6f48-4ef2-a167-370a6610ad8a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:55:25.855986 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.855822 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-data" (OuterVolumeSpecName: "istio-data") pod "929c1f5e-6f48-4ef2-a167-370a6610ad8a" (UID: "929c1f5e-6f48-4ef2-a167-370a6610ad8a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:55:25.856113 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.855994 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "929c1f5e-6f48-4ef2-a167-370a6610ad8a" (UID: "929c1f5e-6f48-4ef2-a167-370a6610ad8a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:55:25.856113 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.856100 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-credential-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:25.856207 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.856118 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istiod-ca-cert\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:25.856207 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.856133 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-workload-certs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:25.856207 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.856144 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-data\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:25.856207 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.856157 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-workload-socket\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:25.857869 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.857834 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/929c1f5e-6f48-4ef2-a167-370a6610ad8a-kube-api-access-bjcd2" (OuterVolumeSpecName: "kube-api-access-bjcd2") pod "929c1f5e-6f48-4ef2-a167-370a6610ad8a" (UID: "929c1f5e-6f48-4ef2-a167-370a6610ad8a"). InnerVolumeSpecName "kube-api-access-bjcd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:55:25.857869 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.857858 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "929c1f5e-6f48-4ef2-a167-370a6610ad8a" (UID: "929c1f5e-6f48-4ef2-a167-370a6610ad8a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:55:25.858027 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.857881 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-token" (OuterVolumeSpecName: "istio-token") pod "929c1f5e-6f48-4ef2-a167-370a6610ad8a" (UID: "929c1f5e-6f48-4ef2-a167-370a6610ad8a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:55:25.858027 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.857876 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "929c1f5e-6f48-4ef2-a167-370a6610ad8a" (UID: "929c1f5e-6f48-4ef2-a167-370a6610ad8a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 23:55:25.956971 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.956936 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-envoy\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:25.956971 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.956966 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-podinfo\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:25.956971 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.956976 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/929c1f5e-6f48-4ef2-a167-370a6610ad8a-istio-token\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:25.957222 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:25.956985 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjcd2\" (UniqueName: \"kubernetes.io/projected/929c1f5e-6f48-4ef2-a167-370a6610ad8a-kube-api-access-bjcd2\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 31 23:55:26.044292 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:26.044253 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc"] Jul 31 23:55:26.049588 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:26.049560 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-3d60f7-openshift-default-6fcd6f88cb-d5tnc"] Jul 31 23:55:26.134268 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:26.134230 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" path="/var/lib/kubelet/pods/929c1f5e-6f48-4ef2-a167-370a6610ad8a/volumes" Jul 31 23:55:26.154051 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:26.154019 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4fhpx_76d59ae5-39ee-42a1-896b-93c7a91c9db1/manager/0.log" Jul 31 23:55:26.723151 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:26.723125 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8567dd98f6-9dfl9_539eb9ef-f6bf-4161-a2fe-e246296d16d8/copy-manifests/0.log" Jul 31 23:55:26.729795 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:26.729768 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8567dd98f6-9dfl9_539eb9ef-f6bf-4161-a2fe-e246296d16d8/manager/0.log" Jul 31 23:55:28.968832 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:28.968793 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-96b8v_ca14563e-f9db-4f8b-832c-73cca13670ae/manager/0.log" Jul 31 23:55:29.191024 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:29.190988 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-qltht_7966c511-43a9-4a69-bd4e-e0b731ca092f/manager/0.log" Jul 31 23:55:29.521071 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:29.521025 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7_86b19028-0f19-4880-ae03-e4654e10fc3d/istio-proxy/0.log" Jul 31 23:55:29.961329 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:29.961289 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-qxx2n_ea5dcba5-3cc5-4452-a308-4c70c86762ce/istio-proxy/1.log" Jul 31 23:55:30.076688 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:30.076655 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4jmd7_472aafd0-2786-4a94-9af0-d433ebed2139/payload-pre-processing/0.log" Jul 31 23:55:30.737632 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:30.737597 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-ds29r_9640ef64-fe7a-4f6e-865f-11b9e494a47f/main/0.log" Jul 31 23:55:30.948759 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:30.948716 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn99g9_c62841fc-d0b7-40e3-a07c-b79e36d0129c/main/0.log" Jul 31 23:55:31.053284 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:31.053178 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-g75n7_580d3d00-c965-4b73-8141-3203a0637ae4/main/0.log" Jul 31 23:55:31.160888 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:31.160829 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-4dcrk_b9c7bbae-1bc8-4c7d-8f44-58bc318989d3/main/0.log" Jul 31 23:55:36.784230 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:36.784190 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:55:36.846305 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:36.846270 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" limit=1024 Jul 31 23:55:39.497530 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:39.497470 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lqgnq_40c30d12-8f96-4d63-a17e-5a3c101f2439/global-pull-secret-syncer/0.log" Jul 31 23:55:39.598425 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:39.598371 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dqrvj_2bd27fa7-9907-419d-af3b-43b77074de67/konnectivity-agent/0.log" Jul 31 23:55:39.718032 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:39.718004 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-148.ec2.internal_a98b158e97d3e00d1e6f3fd6fce59404/haproxy/0.log" Jul 31 23:55:44.833135 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:44.833098 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-96b8v_ca14563e-f9db-4f8b-832c-73cca13670ae/manager/0.log" Jul 31 23:55:45.007639 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:45.007599 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-qltht_7966c511-43a9-4a69-bd4e-e0b731ca092f/manager/0.log" Jul 31 23:55:46.480977 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:46.480948 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0a693b69-6296-4761-a114-7b85f033b766/alertmanager/0.log" Jul 31 23:55:46.502309 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:46.502279 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0a693b69-6296-4761-a114-7b85f033b766/config-reloader/0.log" Jul 31 23:55:46.527677 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:46.527644 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0a693b69-6296-4761-a114-7b85f033b766/kube-rbac-proxy-web/0.log" Jul 31 23:55:46.563340 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:46.563305 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0a693b69-6296-4761-a114-7b85f033b766/kube-rbac-proxy/0.log" Jul 31 23:55:46.592021 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:46.591972 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0a693b69-6296-4761-a114-7b85f033b766/kube-rbac-proxy-metric/0.log" Jul 31 23:55:46.614287 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:46.614179 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0a693b69-6296-4761-a114-7b85f033b766/prom-label-proxy/0.log" Jul 31 23:55:46.639889 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:46.639857 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0a693b69-6296-4761-a114-7b85f033b766/init-config-reloader/0.log" Jul 31 23:55:46.735141 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:46.735109 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-5fcmv_0689778d-1ccf-4e60-a5f0-72ad871a40ba/kube-state-metrics/0.log" Jul 31 23:55:46.753130 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:46.753093 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-5fcmv_0689778d-1ccf-4e60-a5f0-72ad871a40ba/kube-rbac-proxy-main/0.log" Jul 31 23:55:46.776228 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:46.776197 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-5fcmv_0689778d-1ccf-4e60-a5f0-72ad871a40ba/kube-rbac-proxy-self/0.log" Jul 31 23:55:46.803087 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:46.803049 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5c7f464cdb-t2x7g_4ed3765c-6eff-4f01-a894-52b84f544755/metrics-server/0.log" Jul 31 23:55:46.838819 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:46.838787 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-6dffbbd67-4z28j_a16f9d70-421d-414d-aca7-f2ea06d89930/monitoring-plugin/0.log" Jul 31 23:55:47.030919 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.030890 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pj2bq_189420ca-6274-47bb-b106-399b9002d242/node-exporter/0.log" Jul 31 23:55:47.053968 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.053938 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pj2bq_189420ca-6274-47bb-b106-399b9002d242/kube-rbac-proxy/0.log" Jul 31 23:55:47.079609 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.079583 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pj2bq_189420ca-6274-47bb-b106-399b9002d242/init-textfile/0.log" Jul 31 23:55:47.109536 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.109483 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-zdfg5_0f6936db-6831-43bf-a6e4-fd33552c2657/kube-rbac-proxy-main/0.log" Jul 31 23:55:47.130847 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.130760 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-zdfg5_0f6936db-6831-43bf-a6e4-fd33552c2657/kube-rbac-proxy-self/0.log" Jul 31 23:55:47.152934 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.152906 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-zdfg5_0f6936db-6831-43bf-a6e4-fd33552c2657/openshift-state-metrics/0.log" Jul 31 23:55:47.196463 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.196412 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b212a90f-da06-473c-b8e0-68249246af87/prometheus/0.log" Jul 31 23:55:47.218711 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.218679 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b212a90f-da06-473c-b8e0-68249246af87/config-reloader/0.log" Jul 31 23:55:47.242454 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.242422 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b212a90f-da06-473c-b8e0-68249246af87/thanos-sidecar/0.log" Jul 31 23:55:47.264927 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.264898 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b212a90f-da06-473c-b8e0-68249246af87/kube-rbac-proxy-web/0.log" Jul 31 23:55:47.285528 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.285460 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b212a90f-da06-473c-b8e0-68249246af87/kube-rbac-proxy/0.log" Jul 31 23:55:47.310237 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.310205 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b212a90f-da06-473c-b8e0-68249246af87/kube-rbac-proxy-thanos/0.log" Jul 31 23:55:47.336210 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.336185 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b212a90f-da06-473c-b8e0-68249246af87/init-config-reloader/0.log" Jul 31 23:55:47.370196 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.370163 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-vfxp5_a4dee31d-eaee-4012-88bf-e7a6d1d759ad/prometheus-operator/0.log" Jul 31 23:55:47.387791 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.387708 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-vfxp5_a4dee31d-eaee-4012-88bf-e7a6d1d759ad/kube-rbac-proxy/0.log" Jul 31 23:55:47.412886 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.412857 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-66685ff6d4-jfsgj_d14db40d-3234-4eb9-b104-28fc2bae1b12/prometheus-operator-admission-webhook/0.log" Jul 31 23:55:47.442431 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.442399 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c8f4cb785-vhmr9_6c512b95-63f0-4dfb-9bd9-d0f23615e4d4/telemeter-client/0.log" Jul 31 23:55:47.464541 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.464489 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c8f4cb785-vhmr9_6c512b95-63f0-4dfb-9bd9-d0f23615e4d4/reload/0.log" Jul 31 23:55:47.486721 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.486674 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c8f4cb785-vhmr9_6c512b95-63f0-4dfb-9bd9-d0f23615e4d4/kube-rbac-proxy/0.log" Jul 31 23:55:47.529843 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.529813 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fb6f66bf9-9qsd8_3744bbd5-b118-4aeb-a4d1-0453f53b6256/thanos-query/0.log" Jul 31 23:55:47.549672 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.549636 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fb6f66bf9-9qsd8_3744bbd5-b118-4aeb-a4d1-0453f53b6256/kube-rbac-proxy-web/0.log" Jul 31 23:55:47.569228 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.569198 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fb6f66bf9-9qsd8_3744bbd5-b118-4aeb-a4d1-0453f53b6256/kube-rbac-proxy/0.log" Jul 31 23:55:47.590759 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.590729 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fb6f66bf9-9qsd8_3744bbd5-b118-4aeb-a4d1-0453f53b6256/prom-label-proxy/0.log" Jul 31 23:55:47.614003 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.613970 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fb6f66bf9-9qsd8_3744bbd5-b118-4aeb-a4d1-0453f53b6256/kube-rbac-proxy-rules/0.log" Jul 31 23:55:47.633178 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:47.633150 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fb6f66bf9-9qsd8_3744bbd5-b118-4aeb-a4d1-0453f53b6256/kube-rbac-proxy-metrics/0.log" Jul 31 23:55:49.310120 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310087 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v"] Jul 31 23:55:49.310578 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310554 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" containerName="istio-proxy" Jul 31 23:55:49.310637 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310579 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" containerName="istio-proxy" Jul 31 23:55:49.310637 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310591 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f70c2a9-37f5-4df0-9978-dc3c766bd87b" containerName="payload-pre-processing" Jul 31 23:55:49.310637 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310598 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f70c2a9-37f5-4df0-9978-dc3c766bd87b" containerName="payload-pre-processing" Jul 31 23:55:49.310637 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310606 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09b96fe4-4714-4c78-a3f2-0be0520fe0ed" containerName="payload-processing" Jul 31 23:55:49.310637 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310612 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="09b96fe4-4714-4c78-a3f2-0be0520fe0ed" containerName="payload-processing" Jul 31 23:55:49.310637 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310625 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="751e6f22-f48a-476f-bbea-570bd79d0d2e" containerName="istio-proxy" Jul 31 23:55:49.310637 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310630 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="751e6f22-f48a-476f-bbea-570bd79d0d2e" containerName="istio-proxy" Jul 31 23:55:49.310973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310638 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" containerName="istio-proxy" Jul 31 23:55:49.310973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310652 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" containerName="istio-proxy" Jul 31 23:55:49.310973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310667 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="452e15d0-d4e3-4cc0-87eb-26ed6c36f76c" containerName="payload-processing" Jul 31 23:55:49.310973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310673 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="452e15d0-d4e3-4cc0-87eb-26ed6c36f76c" containerName="payload-processing" Jul 31 23:55:49.310973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310680 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea7a873-3f39-46d4-885d-d25f1774d288" containerName="main" Jul 31 23:55:49.310973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310685 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea7a873-3f39-46d4-885d-d25f1774d288" containerName="main" Jul 31 23:55:49.310973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310691 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="751e6f22-f48a-476f-bbea-570bd79d0d2e" containerName="istio-proxy" Jul 31 23:55:49.310973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310695 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="751e6f22-f48a-476f-bbea-570bd79d0d2e" containerName="istio-proxy" Jul 31 23:55:49.310973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310774 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="452e15d0-d4e3-4cc0-87eb-26ed6c36f76c" containerName="payload-processing" Jul 31 23:55:49.310973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310785 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f70c2a9-37f5-4df0-9978-dc3c766bd87b" containerName="payload-pre-processing" Jul 31 23:55:49.310973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310794 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="751e6f22-f48a-476f-bbea-570bd79d0d2e" containerName="istio-proxy" Jul 31 23:55:49.310973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310804 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cea7a873-3f39-46d4-885d-d25f1774d288" containerName="main" Jul 31 23:55:49.310973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310814 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" containerName="istio-proxy" Jul 31 23:55:49.310973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310822 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="929c1f5e-6f48-4ef2-a167-370a6610ad8a" containerName="istio-proxy" Jul 31 23:55:49.310973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.310830 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="09b96fe4-4714-4c78-a3f2-0be0520fe0ed" containerName="payload-processing" Jul 31 23:55:49.314082 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.314065 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.317290 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.317268 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6dw7l\"/\"kube-root-ca.crt\"" Jul 31 23:55:49.317441 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.317292 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6dw7l\"/\"openshift-service-ca.crt\"" Jul 31 23:55:49.318421 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.318401 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6dw7l\"/\"default-dockercfg-vwvnq\"" Jul 31 23:55:49.323183 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.323159 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v"] Jul 31 23:55:49.388971 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.388932 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4-proc\") pod \"perf-node-gather-daemonset-c6d7v\" (UID: \"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.389174 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.389058 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtlnh\" (UniqueName: \"kubernetes.io/projected/baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4-kube-api-access-qtlnh\") pod \"perf-node-gather-daemonset-c6d7v\" (UID: \"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.389174 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.389107 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4-podres\") pod \"perf-node-gather-daemonset-c6d7v\" (UID: \"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.389174 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.389129 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4-sys\") pod \"perf-node-gather-daemonset-c6d7v\" (UID: \"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.389174 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.389145 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4-lib-modules\") pod \"perf-node-gather-daemonset-c6d7v\" (UID: \"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.490464 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.490414 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtlnh\" (UniqueName: \"kubernetes.io/projected/baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4-kube-api-access-qtlnh\") pod \"perf-node-gather-daemonset-c6d7v\" (UID: \"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.490703 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.490488 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4-podres\") pod \"perf-node-gather-daemonset-c6d7v\" (UID: \"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.490703 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.490550 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4-sys\") pod \"perf-node-gather-daemonset-c6d7v\" (UID: \"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.490703 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.490572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4-lib-modules\") pod \"perf-node-gather-daemonset-c6d7v\" (UID: \"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.490703 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.490617 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4-proc\") pod \"perf-node-gather-daemonset-c6d7v\" (UID: \"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.490703 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.490660 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4-podres\") pod \"perf-node-gather-daemonset-c6d7v\" (UID: \"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.490703 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.490667 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4-sys\") pod \"perf-node-gather-daemonset-c6d7v\" (UID: \"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.490967 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.490720 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4-lib-modules\") pod \"perf-node-gather-daemonset-c6d7v\" (UID: \"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.490967 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.490723 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4-proc\") pod \"perf-node-gather-daemonset-c6d7v\" (UID: \"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.499447 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.499417 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtlnh\" (UniqueName: \"kubernetes.io/projected/baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4-kube-api-access-qtlnh\") pod \"perf-node-gather-daemonset-c6d7v\" (UID: \"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.625123 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.625015 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.760801 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.760776 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v"] Jul 31 23:55:49.763160 ip-10-0-131-148 kubenswrapper[2561]: W0731 23:55:49.763119 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbaffb626_b1b1_4fb5_9c6c_ef0b74ff26b4.slice/crio-b5c0c4f8c0e5995200f79c0a365172fbc7446e776e9eaded6423bd645e2b7bb1 WatchSource:0}: Error finding container b5c0c4f8c0e5995200f79c0a365172fbc7446e776e9eaded6423bd645e2b7bb1: Status 404 returned error can't find the container with id b5c0c4f8c0e5995200f79c0a365172fbc7446e776e9eaded6423bd645e2b7bb1 Jul 31 23:55:49.856764 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.856734 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" event={"ID":"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4","Type":"ContainerStarted","Data":"00c906dabec16a9ac2468a2eea13186dbddc48372e3abdde8c09ab970ff4c339"} Jul 31 23:55:49.856903 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.856786 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" event={"ID":"baffb626-b1b1-4fb5-9c6c-ef0b74ff26b4","Type":"ContainerStarted","Data":"b5c0c4f8c0e5995200f79c0a365172fbc7446e776e9eaded6423bd645e2b7bb1"} Jul 31 23:55:49.856903 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.856842 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:49.873803 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:49.873749 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" podStartSLOduration=0.873733956 podStartE2EDuration="873.733956ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:55:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:55:49.871304007 +0000 UTC m=+3506.343748189" watchObservedRunningTime="2026-07-31 23:55:49.873733956 +0000 UTC m=+3506.346178155" Jul 31 23:55:51.905631 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:51.905597 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zc76w_b266c17f-9ecf-42a8-8843-4699f8a0cca5/dns/0.log" Jul 31 23:55:51.924947 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:51.924916 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zc76w_b266c17f-9ecf-42a8-8843-4699f8a0cca5/kube-rbac-proxy/0.log" Jul 31 23:55:51.945519 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:51.945478 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5t5xj_5753fb43-dec6-43b9-806e-3c593edb98dd/dns-node-resolver/0.log" Jul 31 23:55:52.540137 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:52.540101 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lzdpn_d85e5451-3e8a-46bb-beb6-883353da3494/node-ca/0.log" Jul 31 23:55:53.481493 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:53.481464 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7_86b19028-0f19-4880-ae03-e4654e10fc3d/istio-proxy/0.log" Jul 31 23:55:53.716584 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:53.716546 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-qxx2n_ea5dcba5-3cc5-4452-a308-4c70c86762ce/istio-proxy/0.log" Jul 31 23:55:53.728923 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:53.728888 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-qxx2n_ea5dcba5-3cc5-4452-a308-4c70c86762ce/istio-proxy/1.log" Jul 31 23:55:53.765804 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:53.765679 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4jmd7_472aafd0-2786-4a94-9af0-d433ebed2139/payload-pre-processing/0.log" Jul 31 23:55:54.612924 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:54.612889 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-prsgk_afb3784e-7558-4f93-9633-3423b9cbf971/serve-healthcheck-canary/0.log" Jul 31 23:55:55.062931 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:55.062895 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-km7l9_a18cb43c-48f7-4bb1-853a-ad278519677f/kube-rbac-proxy/0.log" Jul 31 23:55:55.082131 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:55.082101 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-km7l9_a18cb43c-48f7-4bb1-853a-ad278519677f/exporter/0.log" Jul 31 23:55:55.102914 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:55.102887 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-km7l9_a18cb43c-48f7-4bb1-853a-ad278519677f/extractor/0.log" Jul 31 23:55:55.872432 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:55.872398 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-c6d7v" Jul 31 23:55:57.287382 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:57.287342 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4fhpx_76d59ae5-39ee-42a1-896b-93c7a91c9db1/manager/0.log" Jul 31 23:55:57.808149 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:57.808112 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8567dd98f6-9dfl9_539eb9ef-f6bf-4161-a2fe-e246296d16d8/manager/0.log" Jul 31 23:55:57.833013 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:57.832984 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8567dd98f6-9dfl9_539eb9ef-f6bf-4161-a2fe-e246296d16d8/copy-manifests/0.log" Jul 31 23:55:59.461997 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:59.461964 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6b79896bff-bbcbp_1930ccb7-2f6c-45d6-a910-c880719b32f3/manager/0.log" Jul 31 23:55:59.510767 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:59.510737 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-88zcc_70848628-570d-4c14-96bc-2163751b3bdb/openshift-lws-operator/0.log" Jul 31 23:55:59.828973 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:55:59.828943 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7_86b19028-0f19-4880-ae03-e4654e10fc3d/istio-proxy/0.log" Jul 31 23:56:00.036188 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:00.036145 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-qxx2n_ea5dcba5-3cc5-4452-a308-4c70c86762ce/istio-proxy/0.log" Jul 31 23:56:00.048980 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:00.048948 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-qxx2n_ea5dcba5-3cc5-4452-a308-4c70c86762ce/istio-proxy/1.log" Jul 31 23:56:00.075954 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:00.075919 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4jmd7_472aafd0-2786-4a94-9af0-d433ebed2139/payload-pre-processing/0.log" Jul 31 23:56:01.496047 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:01.496010 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cb7vt7_86b19028-0f19-4880-ae03-e4654e10fc3d/istio-proxy/0.log" Jul 31 23:56:01.714935 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:01.714876 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-qxx2n_ea5dcba5-3cc5-4452-a308-4c70c86762ce/istio-proxy/0.log" Jul 31 23:56:01.728019 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:01.727987 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-qxx2n_ea5dcba5-3cc5-4452-a308-4c70c86762ce/istio-proxy/1.log" Jul 31 23:56:01.761370 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:01.761271 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4jmd7_472aafd0-2786-4a94-9af0-d433ebed2139/payload-pre-processing/0.log" Jul 31 23:56:05.810821 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:05.810792 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p6gvt_4c51d9ed-66ce-4160-8f08-bc20664c47b5/kube-multus-additional-cni-plugins/0.log" Jul 31 23:56:05.830272 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:05.830242 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p6gvt_4c51d9ed-66ce-4160-8f08-bc20664c47b5/egress-router-binary-copy/0.log" Jul 31 23:56:05.849024 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:05.849001 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p6gvt_4c51d9ed-66ce-4160-8f08-bc20664c47b5/cni-plugins/0.log" Jul 31 23:56:05.870560 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:05.870532 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p6gvt_4c51d9ed-66ce-4160-8f08-bc20664c47b5/bond-cni-plugin/0.log" Jul 31 23:56:05.891072 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:05.891037 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p6gvt_4c51d9ed-66ce-4160-8f08-bc20664c47b5/routeoverride-cni/0.log" Jul 31 23:56:05.909781 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:05.909753 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p6gvt_4c51d9ed-66ce-4160-8f08-bc20664c47b5/whereabouts-cni-bincopy/0.log" Jul 31 23:56:05.928695 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:05.928667 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p6gvt_4c51d9ed-66ce-4160-8f08-bc20664c47b5/whereabouts-cni/0.log" Jul 31 23:56:06.038312 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:06.038273 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-sv97r_02a7725c-7910-4311-bba9-dacdbc0d7ce5/kube-multus/0.log" Jul 31 23:56:06.061297 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:06.061205 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-44zk2_58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e/network-metrics-daemon/0.log" Jul 31 23:56:06.078215 ip-10-0-131-148 kubenswrapper[2561]: I0731 23:56:06.078187 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-44zk2_58daa0fe-d3d1-4c52-8d5e-94bc5c3f5b3e/kube-rbac-proxy/0.log"