Jul 28 19:36:44.179946 ip-10-0-136-194 systemd[1]: Starting Kubernetes Kubelet... Jul 28 19:36:44.604609 ip-10-0-136-194 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 19:36:44.604609 ip-10-0-136-194 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 28 19:36:44.604609 ip-10-0-136-194 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 19:36:44.604609 ip-10-0-136-194 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 28 19:36:44.604609 ip-10-0-136-194 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 19:36:44.606486 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.606331 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 28 19:36:44.614121 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614102 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 19:36:44.614121 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614121 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614125 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614129 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614132 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614136 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614139 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614142 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614145 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614147 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614155 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614160 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614164 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614168 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614171 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614175 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614178 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614181 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614184 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614188 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 19:36:44.614200 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614191 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614194 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614197 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614199 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614202 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614205 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614208 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614211 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614215 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614218 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614221 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614225 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614228 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614230 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614233 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614236 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614238 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614241 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614244 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614247 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 19:36:44.614692 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614249 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614252 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614255 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614257 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614260 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614262 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614265 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614268 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614271 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614274 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614276 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614279 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614289 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614291 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614295 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614299 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614302 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614305 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614308 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614311 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 19:36:44.615261 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614313 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614316 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614319 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614322 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614325 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614327 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614330 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614334 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614336 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614339 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614341 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614344 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614347 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614350 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614354 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614358 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614361 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614363 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614366 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 19:36:44.615804 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614369 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614371 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614374 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614377 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614380 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614388 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614398 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614402 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614405 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614408 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614411 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614413 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614416 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614419 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614422 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614424 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614427 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614430 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614432 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.614435 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 19:36:44.616308 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615220 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615229 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615233 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615237 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615240 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615243 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615246 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615249 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615252 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615255 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615258 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615261 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615264 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615268 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615272 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615276 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615279 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615282 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615285 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 19:36:44.616856 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615290 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615293 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615296 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615298 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615301 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615304 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615307 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615310 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615313 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615315 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615318 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615321 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615324 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615326 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615329 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615332 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615334 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615338 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615341 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615343 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 19:36:44.617360 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615346 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615349 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615351 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615354 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615356 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615359 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615361 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615364 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615367 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615369 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615372 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615375 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615378 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615382 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615385 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615388 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615391 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615393 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615396 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615399 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 19:36:44.617907 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615402 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615404 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615407 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615409 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615412 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615415 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615418 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615420 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615423 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615425 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615428 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615431 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615433 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615436 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615438 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615441 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615444 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615447 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615449 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615452 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 19:36:44.618474 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615456 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615460 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615464 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615467 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615473 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615477 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615480 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615483 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615486 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615489 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615492 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615495 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615497 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615500 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615503 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615505 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615509 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615512 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615514 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 19:36:44.619003 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.615517 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616015 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616029 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616033 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616036 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616040 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616043 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616046 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616049 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616052 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616055 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616059 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616062 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616065 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616070 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616075 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616078 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616082 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616085 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 19:36:44.619502 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616088 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616091 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616094 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616097 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616100 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616103 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616106 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616110 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616113 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616116 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616119 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616123 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616125 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616129 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616132 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616136 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616139 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616141 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616144 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616148 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 19:36:44.620022 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616150 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616153 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616156 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616159 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616161 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616164 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616167 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616170 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616172 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616175 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616178 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616181 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616183 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616186 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616189 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616193 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616198 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616201 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616205 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 19:36:44.620569 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616208 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616211 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616214 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616216 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616219 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616222 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616225 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616229 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616232 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616235 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616238 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616241 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616244 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616248 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616251 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616254 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616257 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616260 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616262 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616265 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 19:36:44.621098 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616268 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616271 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616274 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616276 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616279 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616281 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616284 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616287 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616290 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616292 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616295 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616303 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616306 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616309 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616312 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616315 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616318 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616322 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616325 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 19:36:44.621634 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616329 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616333 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.616336 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616412 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616420 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616428 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616433 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616438 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616442 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616447 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616452 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616455 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616459 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616463 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616467 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616470 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616473 2564 flags.go:64] FLAG: --cgroup-root="" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616477 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616480 2564 flags.go:64] FLAG: --client-ca-file="" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616483 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616486 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616491 2564 flags.go:64] FLAG: --cluster-domain="" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616494 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616497 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 28 19:36:44.622149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616503 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616507 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616511 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616515 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616518 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616522 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616525 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616528 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616532 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616535 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616538 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616543 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616547 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616550 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616553 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616556 2564 flags.go:64] FLAG: --enable-server="true" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616560 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616565 2564 flags.go:64] FLAG: --event-burst="100" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616568 2564 flags.go:64] FLAG: --event-qps="50" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616571 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616574 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616578 2564 flags.go:64] FLAG: --eviction-hard="" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616586 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616589 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616592 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 28 19:36:44.622785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616595 2564 flags.go:64] FLAG: --eviction-soft="" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616598 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616602 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616605 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616609 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616612 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616615 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616620 2564 flags.go:64] FLAG: --feature-gates="" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616624 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616627 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616631 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616634 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616637 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616640 2564 flags.go:64] FLAG: --help="false" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616644 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-136-194.ec2.internal" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616655 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616660 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616663 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616667 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616671 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616675 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616678 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616682 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 28 19:36:44.623430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616685 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616688 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616692 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616695 2564 flags.go:64] FLAG: --kube-reserved="" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616698 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616715 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616719 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616722 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616725 2564 flags.go:64] FLAG: --lock-file="" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616728 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616731 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616734 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616740 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616744 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616747 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616750 2564 flags.go:64] FLAG: --logging-format="text" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616755 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616759 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616763 2564 flags.go:64] FLAG: --manifest-url="" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616766 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616771 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616775 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616779 2564 flags.go:64] FLAG: --max-pods="110" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616783 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616786 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 28 19:36:44.624082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616790 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616793 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616796 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616800 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616803 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616819 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616822 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616826 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616829 2564 flags.go:64] FLAG: --pod-cidr="" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616832 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616838 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616841 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616844 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616847 2564 flags.go:64] FLAG: --port="10250" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616851 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616854 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05629675b4e62ab26" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616857 2564 flags.go:64] FLAG: --qos-reserved="" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616861 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616864 2564 flags.go:64] FLAG: --register-node="true" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616867 2564 flags.go:64] FLAG: --register-with-taints="" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616872 2564 flags.go:64] FLAG: --registry-burst="10" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616875 2564 flags.go:64] FLAG: --registry-qps="5" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616878 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616882 2564 flags.go:64] FLAG: --reserved-memory="" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616886 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 28 19:36:44.624753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616889 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616893 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616896 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616899 2564 flags.go:64] FLAG: --runonce="false" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616902 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616905 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616909 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616912 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616915 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616918 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616921 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616925 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616927 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616937 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616940 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616943 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616946 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616949 2564 flags.go:64] FLAG: --system-cgroups="" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616952 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616958 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616961 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616964 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616968 2564 flags.go:64] FLAG: --tls-min-version="" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616971 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616974 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 28 19:36:44.625419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616977 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616980 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616984 2564 flags.go:64] FLAG: --v="2" Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616988 2564 flags.go:64] FLAG: --version="false" Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616992 2564 flags.go:64] FLAG: --vmodule="" Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.616997 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.617001 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617195 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617201 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617205 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617208 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617212 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617214 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617217 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617220 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617223 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617226 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617229 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617232 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617235 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617238 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 19:36:44.626609 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617248 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617253 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617257 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617260 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617263 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617266 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617269 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617272 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617274 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617277 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617280 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617282 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617285 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617288 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617291 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617294 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617297 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617302 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617305 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617308 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 19:36:44.627431 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617310 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617313 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617316 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617319 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617322 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617324 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617327 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617330 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617333 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617335 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617338 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617341 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617344 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617353 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617356 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617359 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617362 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617365 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617368 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617372 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 19:36:44.627982 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617375 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617378 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617380 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617383 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617386 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617388 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617391 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617394 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617398 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617403 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617406 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617408 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617411 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617414 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617417 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617420 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617423 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617426 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617428 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 28 19:36:44.628542 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617431 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617434 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617437 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617440 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617442 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617445 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617448 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617459 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617463 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617466 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617470 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617474 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617477 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617480 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617483 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617486 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617489 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617491 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617494 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 19:36:44.629114 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617497 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 19:36:44.629632 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617502 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 19:36:44.629632 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617505 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 19:36:44.629632 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617510 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 19:36:44.629632 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617512 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 19:36:44.629632 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617516 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 19:36:44.629632 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.617518 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 19:36:44.629632 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.618147 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 19:36:44.629632 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.627519 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 28 19:36:44.629632 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.627539 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 28 19:36:44.629632 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.627561 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 28 19:36:44.629632 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.627568 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 28 19:36:44.629632 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627633 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 19:36:44.629632 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627639 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 19:36:44.629632 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627643 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 19:36:44.629632 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627646 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627649 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627653 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627655 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627658 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627661 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627664 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627667 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627671 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627673 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627676 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627679 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627687 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627690 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627693 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627696 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627699 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627702 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627705 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 19:36:44.630071 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627726 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627731 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627734 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627737 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627741 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627744 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627747 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627751 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627754 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627757 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627761 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627764 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627768 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627772 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627775 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627778 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627781 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627784 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627787 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627790 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 19:36:44.630574 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627792 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627795 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627798 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627801 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627812 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627815 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627818 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627820 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627823 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627826 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627828 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627831 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627834 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627837 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627840 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627842 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627846 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627850 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627853 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 19:36:44.631232 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627857 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627860 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627863 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627866 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627869 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627871 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627874 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627877 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627879 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627882 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627885 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627888 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627891 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627893 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627896 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627899 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627901 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627904 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627912 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627915 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 19:36:44.631753 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627917 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627920 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627923 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627926 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627929 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627932 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627934 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627937 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627940 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627943 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627947 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627949 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627953 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627956 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627959 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627962 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627964 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 19:36:44.632279 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.627967 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 19:36:44.632779 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.627973 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 19:36:44.632779 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628115 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 19:36:44.632779 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628119 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 19:36:44.632779 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628123 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 19:36:44.632779 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628126 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 19:36:44.632779 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628129 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 19:36:44.632779 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628132 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 19:36:44.632779 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628135 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 19:36:44.632779 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628137 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 19:36:44.632779 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628140 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 19:36:44.632779 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628143 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 19:36:44.632779 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628146 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 19:36:44.632779 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628149 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 19:36:44.632779 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628157 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628160 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628163 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628166 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628169 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628171 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628174 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628177 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628180 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628182 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628185 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628189 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628193 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628196 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628200 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628203 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628207 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628210 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628213 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 19:36:44.633149 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628216 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628219 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628222 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628225 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628228 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628230 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628233 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628236 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628239 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628242 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628245 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628249 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628252 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628255 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628263 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628267 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628270 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628273 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628276 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628279 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 19:36:44.633656 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628282 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628285 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628287 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628290 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628293 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628295 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628298 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628302 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628305 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628307 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628310 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628313 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628315 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628318 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628320 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628323 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628326 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628329 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628331 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628334 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 19:36:44.634211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628336 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628339 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628342 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628344 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628347 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628349 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628352 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628362 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628365 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628368 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628370 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628373 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628376 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628379 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628382 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628384 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628387 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628392 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628395 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 19:36:44.634755 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628398 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 19:36:44.635253 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628401 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 19:36:44.635253 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628404 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 19:36:44.635253 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628407 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 19:36:44.635253 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628410 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 19:36:44.635253 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628412 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 19:36:44.635253 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628415 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 19:36:44.635253 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628417 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 19:36:44.635253 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:44.628421 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 19:36:44.635253 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.628426 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 19:36:44.635253 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.628560 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 28 19:36:44.635253 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.631155 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 28 19:36:44.635253 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.632737 2564 server.go:1019] "Starting client certificate rotation" Jul 28 19:36:44.635253 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.632851 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 19:36:44.635253 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.632889 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 19:36:44.635645 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.635611 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 19:36:44.638366 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.638348 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 19:36:44.647918 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.647896 2564 log.go:25] "Validated CRI v1 runtime API" Jul 28 19:36:44.652437 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.652418 2564 log.go:25] "Validated CRI v1 image API" Jul 28 19:36:44.653393 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.653371 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 28 19:36:44.657294 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.657266 2564 fs.go:135] Filesystem UUIDs: map[1c2e73a0-545e-4b80-8a6d-2d2fabb10de0:/dev/nvme0n1p3 1c334616-926b-4fd6-9d7c-2caa0bf4285b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 28 19:36:44.657294 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.657287 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 28 19:36:44.664399 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.664269 2564 manager.go:216] Machine: {Timestamp:2026-07-28 19:36:44.662784904 +0000 UTC m=+0.369996812 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098625 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec296cdf97988b90f313dfa7d7506189 SystemUUID:ec296cdf-9798-8b90-f313-dfa7d7506189 BootID:5a9c40e9-2ef3-440a-aed2-59a6ad6465c5 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:33:fe:b2:63:63 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:33:fe:b2:63:63 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:32:b9:90:8b:45 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 28 19:36:44.664399 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.664390 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 28 19:36:44.664523 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.664480 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 28 19:36:44.665230 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.665213 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 19:36:44.667941 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.667910 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 28 19:36:44.668092 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.667946 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-194.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 28 19:36:44.668888 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.668877 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 28 19:36:44.668925 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.668891 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 28 19:36:44.668925 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.668899 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 19:36:44.669799 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.669788 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 19:36:44.670746 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.670736 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 28 19:36:44.672561 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.672548 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 28 19:36:44.672693 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.672682 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 28 19:36:44.672745 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.672728 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 28 19:36:44.672745 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.672741 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 28 19:36:44.672804 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.672759 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 28 19:36:44.672804 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.672770 2564 kubelet.go:398] "Adding apiserver pod source" Jul 28 19:36:44.672804 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.672780 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 28 19:36:44.673233 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.673214 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 19:36:44.673281 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.673254 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 19:36:44.676988 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.676965 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 28 19:36:44.679014 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.678998 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 28 19:36:44.679086 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.679033 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 28 19:36:44.680637 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.680624 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 28 19:36:44.680675 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.680641 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 28 19:36:44.680675 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.680647 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 28 19:36:44.680675 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.680654 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 28 19:36:44.680675 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.680663 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 28 19:36:44.680675 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.680669 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 28 19:36:44.680675 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.680678 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 28 19:36:44.680871 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.680684 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 28 19:36:44.680871 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.680692 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 28 19:36:44.680871 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.680697 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 28 19:36:44.680871 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.680706 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 28 19:36:44.680871 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.680725 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 28 19:36:44.681341 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.681331 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 28 19:36:44.681341 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.681341 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 28 19:36:44.684491 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:44.684465 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-194.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 28 19:36:44.684491 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:44.684463 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 28 19:36:44.684612 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.684475 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-194.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 28 19:36:44.685221 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.685203 2564 server.go:1268] "Started kubelet" Jul 28 19:36:44.685334 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.685278 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 28 19:36:44.685394 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.685307 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 28 19:36:44.685394 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.685378 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 28 19:36:44.686083 ip-10-0-136-194 systemd[1]: Started Kubernetes Kubelet. Jul 28 19:36:44.687443 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.687425 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 28 19:36:44.688101 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.688084 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 28 19:36:44.691790 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:44.690707 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-194.ec2.internal.18c68b3eed57e6c5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-194.ec2.internal,UID:ip-10-0-136-194.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-194.ec2.internal,},FirstTimestamp:2026-07-28 19:36:44.685174469 +0000 UTC m=+0.392386357,LastTimestamp:2026-07-28 19:36:44.685174469 +0000 UTC m=+0.392386357,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-194.ec2.internal,}" Jul 28 19:36:44.691910 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.691806 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4bslv" Jul 28 19:36:44.692193 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:44.692160 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 28 19:36:44.695453 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.695431 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 28 19:36:44.695453 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.695440 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 28 19:36:44.695596 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:44.695577 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 28 19:36:44.696303 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.696281 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 28 19:36:44.696424 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.696303 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 28 19:36:44.696424 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.696425 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 28 19:36:44.696598 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.696353 2564 factory.go:55] Registering systemd factory Jul 28 19:36:44.696598 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.696506 2564 factory.go:223] Registration of the systemd container factory successfully Jul 28 19:36:44.696598 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.696532 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 28 19:36:44.696598 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.696540 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 28 19:36:44.696806 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.696723 2564 factory.go:153] Registering CRI-O factory Jul 28 19:36:44.696806 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.696739 2564 factory.go:223] Registration of the crio container factory successfully Jul 28 19:36:44.696806 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.696795 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 28 19:36:44.696943 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.696818 2564 factory.go:103] Registering Raw factory Jul 28 19:36:44.696943 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.696829 2564 manager.go:1196] Started watching for new ooms in manager Jul 28 19:36:44.697375 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.697360 2564 manager.go:349] Starting recovery of all containers Jul 28 19:36:44.700581 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.700555 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4bslv" Jul 28 19:36:44.705434 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.705397 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 28 19:36:44.708771 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.708748 2564 manager.go:354] Recovery completed Jul 28 19:36:44.709175 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:44.707856 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-194.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 28 19:36:44.709260 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:44.709203 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 28 19:36:44.712107 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:44.711964 2564 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 28 19:36:44.715245 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.715233 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 19:36:44.717469 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.717451 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientMemory" Jul 28 19:36:44.717545 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.717481 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 19:36:44.717545 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.717492 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientPID" Jul 28 19:36:44.718087 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.718071 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 28 19:36:44.718087 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.718087 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 19:36:44.718215 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.718088 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 28 19:36:44.718215 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.718108 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 28 19:36:44.720132 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.720119 2564 policy_none.go:49] "None policy: Start" Jul 28 19:36:44.720172 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.720136 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 28 19:36:44.720172 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.720147 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 28 19:36:44.721089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.721078 2564 policy_none.go:47] "Start" Jul 28 19:36:44.722790 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.722773 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientMemory" Jul 28 19:36:44.722883 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.722806 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 19:36:44.722883 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.722821 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientPID" Jul 28 19:36:44.757310 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.757295 2564 manager.go:337] "Starting Device Plugin manager" Jul 28 19:36:44.757416 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:44.757328 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 28 19:36:44.757416 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.757339 2564 server.go:87] "Starting device plugin registration server" Jul 28 19:36:44.757602 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.757589 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 28 19:36:44.757639 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.757607 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 28 19:36:44.757807 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.757789 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 28 19:36:44.757883 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.757869 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 28 19:36:44.757883 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.757879 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 28 19:36:44.758750 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:44.758665 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 28 19:36:44.758750 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:44.758721 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-194.ec2.internal\" not found" Jul 28 19:36:44.858402 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.858334 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 19:36:44.859608 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.859400 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientMemory" Jul 28 19:36:44.859608 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.859439 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 19:36:44.859608 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.859456 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientPID" Jul 28 19:36:44.859608 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.859497 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-194.ec2.internal" Jul 28 19:36:44.868687 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.868667 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-194.ec2.internal" Jul 28 19:36:44.868803 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:44.868691 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-194.ec2.internal\": node \"ip-10-0-136-194.ec2.internal\" not found" Jul 28 19:36:44.887171 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:44.887149 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 28 19:36:44.912058 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.912037 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 28 19:36:44.912154 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.912077 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 28 19:36:44.912154 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.912101 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 28 19:36:44.912154 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:44.912132 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 28 19:36:44.914315 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:44.914299 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 19:36:44.987361 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:44.987318 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 28 19:36:45.012569 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.012524 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal"] Jul 28 19:36:45.012669 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.012616 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 19:36:45.013713 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.013689 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientMemory" Jul 28 19:36:45.013842 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.013726 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 19:36:45.013842 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.013741 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientPID" Jul 28 19:36:45.015170 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.015155 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 19:36:45.015287 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.015272 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 28 19:36:45.015344 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.015304 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 19:36:45.015951 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.015932 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientMemory" Jul 28 19:36:45.016041 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.015968 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 19:36:45.016041 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.015980 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientPID" Jul 28 19:36:45.016041 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.015932 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientMemory" Jul 28 19:36:45.016142 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.016046 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 19:36:45.016142 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.016065 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientPID" Jul 28 19:36:45.017276 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.017261 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" Jul 28 19:36:45.017344 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.017284 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 19:36:45.017961 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.017944 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientMemory" Jul 28 19:36:45.018058 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.017972 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 19:36:45.018058 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.017985 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientPID" Jul 28 19:36:45.035274 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:45.035255 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-194.ec2.internal\" not found" node="ip-10-0-136-194.ec2.internal" Jul 28 19:36:45.040231 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:45.040211 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-194.ec2.internal\" not found" node="ip-10-0-136-194.ec2.internal" Jul 28 19:36:45.087924 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:45.087899 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 28 19:36:45.097941 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.097917 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/06f51ddf0048f10e09fab733759c2c7c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal\" (UID: \"06f51ddf0048f10e09fab733759c2c7c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 28 19:36:45.097997 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.097949 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06f51ddf0048f10e09fab733759c2c7c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal\" (UID: \"06f51ddf0048f10e09fab733759c2c7c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 28 19:36:45.097997 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.097968 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1eb1ec1d2f5d49622c78b5890bf891f2-config\") pod \"kube-apiserver-proxy-ip-10-0-136-194.ec2.internal\" (UID: \"1eb1ec1d2f5d49622c78b5890bf891f2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" Jul 28 19:36:45.188391 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:45.188314 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 28 19:36:45.199207 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.199187 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/06f51ddf0048f10e09fab733759c2c7c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal\" (UID: \"06f51ddf0048f10e09fab733759c2c7c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 28 19:36:45.199262 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.199212 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06f51ddf0048f10e09fab733759c2c7c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal\" (UID: \"06f51ddf0048f10e09fab733759c2c7c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 28 19:36:45.199262 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.199231 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1eb1ec1d2f5d49622c78b5890bf891f2-config\") pod \"kube-apiserver-proxy-ip-10-0-136-194.ec2.internal\" (UID: \"1eb1ec1d2f5d49622c78b5890bf891f2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" Jul 28 19:36:45.199328 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.199261 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1eb1ec1d2f5d49622c78b5890bf891f2-config\") pod \"kube-apiserver-proxy-ip-10-0-136-194.ec2.internal\" (UID: \"1eb1ec1d2f5d49622c78b5890bf891f2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" Jul 28 19:36:45.199328 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.199288 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06f51ddf0048f10e09fab733759c2c7c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal\" (UID: \"06f51ddf0048f10e09fab733759c2c7c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 28 19:36:45.199328 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.199289 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/06f51ddf0048f10e09fab733759c2c7c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal\" (UID: \"06f51ddf0048f10e09fab733759c2c7c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 28 19:36:45.292184 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:45.292135 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 28 19:36:45.340280 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.340238 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 28 19:36:45.343388 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.343366 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" Jul 28 19:36:45.393288 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:45.393257 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 28 19:36:45.500454 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:45.500382 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 28 19:36:45.602256 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:45.602215 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 28 19:36:45.631843 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.631810 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 28 19:36:45.632480 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.631852 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 19:36:45.632480 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.631934 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 19:36:45.703242 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:45.703208 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 28 19:36:45.710589 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.710553 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 19:31:45 +0000 UTC" deadline="2028-02-13 19:54:16.037661191 +0000 UTC" Jul 28 19:36:45.710589 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.710586 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13560h17m30.327078051s" Jul 28 19:36:45.804361 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:45.804335 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 28 19:36:45.896734 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:45.896693 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06f51ddf0048f10e09fab733759c2c7c.slice/crio-696ff818e67c1075aa0c382cccd4c37fe846595b51e3e1e1a6df2a75a6dd7ad9 WatchSource:0}: Error finding container 696ff818e67c1075aa0c382cccd4c37fe846595b51e3e1e1a6df2a75a6dd7ad9: Status 404 returned error can't find the container with id 696ff818e67c1075aa0c382cccd4c37fe846595b51e3e1e1a6df2a75a6dd7ad9 Jul 28 19:36:45.904733 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:45.904698 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 28 19:36:45.904786 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.904761 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 19:36:45.916075 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.916033 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" event={"ID":"06f51ddf0048f10e09fab733759c2c7c","Type":"ContainerStarted","Data":"696ff818e67c1075aa0c382cccd4c37fe846595b51e3e1e1a6df2a75a6dd7ad9"} Jul 28 19:36:45.976446 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.976422 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 19:36:45.996467 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:45.996439 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 28 19:36:46.010641 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.010621 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 19:36:46.011606 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.011590 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" Jul 28 19:36:46.021173 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.021153 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 19:36:46.042310 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:46.042286 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eb1ec1d2f5d49622c78b5890bf891f2.slice/crio-7bee2b64bdc8b3e1840c7848d303a89fb9a06644f8338e3f5eeff3784d3596cf WatchSource:0}: Error finding container 7bee2b64bdc8b3e1840c7848d303a89fb9a06644f8338e3f5eeff3784d3596cf: Status 404 returned error can't find the container with id 7bee2b64bdc8b3e1840c7848d303a89fb9a06644f8338e3f5eeff3784d3596cf Jul 28 19:36:46.200832 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.200747 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 19:36:46.675960 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.675578 2564 apiserver.go:52] "Watching apiserver" Jul 28 19:36:46.689860 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.689833 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 28 19:36:46.690861 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.690827 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-mq2zb","openshift-image-registry/node-ca-6kkrr","openshift-multus/multus-additional-cni-plugins-gpbk6","openshift-multus/multus-tj7qj","openshift-multus/network-metrics-daemon-q6qbg","openshift-network-diagnostics/network-check-target-p4mcm","kube-system/global-pull-secret-syncer-2xb59","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4","openshift-dns/node-resolver-s2jhz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal","openshift-network-operator/iptables-alerter-s7ftg","openshift-ovn-kubernetes/ovnkube-node-vl4ft","kube-system/konnectivity-agent-b7h24","kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal"] Jul 28 19:36:46.693557 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.693532 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:46.693758 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:46.693631 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:36:46.695275 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.694833 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b7h24" Jul 28 19:36:46.696909 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.696587 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.696909 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.696743 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 28 19:36:46.701200 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.700551 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.701200 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.701031 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:46.701200 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:46.701094 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:36:46.703400 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.703374 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-m7n6n\"" Jul 28 19:36:46.703716 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.703691 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 28 19:36:46.703937 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.703922 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 28 19:36:46.704356 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.704334 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 28 19:36:46.704443 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.704377 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 28 19:36:46.704536 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.704520 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:36:46.704603 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:46.704580 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:36:46.704669 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.704654 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 28 19:36:46.704805 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.704665 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s2jhz" Jul 28 19:36:46.704805 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.704777 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 28 19:36:46.706234 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.705209 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 28 19:36:46.706234 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.704334 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-l9qnp\"" Jul 28 19:36:46.706234 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.705409 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.706234 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.705440 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 28 19:36:46.706234 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.705523 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rld2f\"" Jul 28 19:36:46.706234 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.705860 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6kkrr" Jul 28 19:36:46.707284 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.707259 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.707367 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.707343 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 28 19:36:46.708559 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.708495 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s7ftg" Jul 28 19:36:46.709541 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.709468 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 28 19:36:46.709541 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.709490 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 28 19:36:46.709744 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.709734 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 28 19:36:46.709860 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.709832 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e333f58-159f-4604-aefa-55d3aeb458b6-cnibin\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.709919 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.709867 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5e23e056-ad6d-42d0-a5b2-54ebc60e3a9c-konnectivity-ca\") pod \"konnectivity-agent-b7h24\" (UID: \"5e23e056-ad6d-42d0-a5b2-54ebc60e3a9c\") " pod="kube-system/konnectivity-agent-b7h24" Jul 28 19:36:46.709919 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.709895 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-run-netns\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.710035 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.709921 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-multus-conf-dir\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.710035 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.709955 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-multus-daemon-config\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.710035 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.709964 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 28 19:36:46.710035 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.709981 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs\") pod \"network-metrics-daemon-q6qbg\" (UID: \"842143d0-262e-4779-90aa-50bb8b33633b\") " pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:46.710035 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710006 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdwfh\" (UniqueName: \"kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh\") pod \"network-check-target-p4mcm\" (UID: \"7442e5a0-6ae1-43a9-b67e-ada4e63d3a41\") " pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:36:46.710035 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710030 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-os-release\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.710327 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710054 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-hostroot\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.710327 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710080 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh72v\" (UniqueName: \"kubernetes.io/projected/9e333f58-159f-4604-aefa-55d3aeb458b6-kube-api-access-wh72v\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.710327 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710107 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:46.710327 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710133 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9e333f58-159f-4604-aefa-55d3aeb458b6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.710327 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710148 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bwpl9\"" Jul 28 19:36:46.710327 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710159 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-cnibin\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.710327 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710200 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e333f58-159f-4604-aefa-55d3aeb458b6-cni-binary-copy\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.710327 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710226 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-run-k8s-cni-cncf-io\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.710327 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710263 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9hdh\" (UniqueName: \"kubernetes.io/projected/842143d0-262e-4779-90aa-50bb8b33633b-kube-api-access-r9hdh\") pod \"network-metrics-daemon-q6qbg\" (UID: \"842143d0-262e-4779-90aa-50bb8b33633b\") " pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:46.710327 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710289 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e437f25-b6ba-42b8-b1c1-9245e8a86bc4-hosts-file\") pod \"node-resolver-s2jhz\" (UID: \"7e437f25-b6ba-42b8-b1c1-9245e8a86bc4\") " pod="openshift-dns/node-resolver-s2jhz" Jul 28 19:36:46.710327 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710315 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b5208c81-86ee-4635-9dd3-137393805762-kubelet-config\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710340 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9e333f58-159f-4604-aefa-55d3aeb458b6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710367 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-cni-binary-copy\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710382 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710390 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-var-lib-cni-bin\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710414 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710434 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-etc-kubernetes\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710459 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-var-lib-kubelet\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710484 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdvgn\" (UniqueName: \"kubernetes.io/projected/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-kube-api-access-mdvgn\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710508 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e437f25-b6ba-42b8-b1c1-9245e8a86bc4-tmp-dir\") pod \"node-resolver-s2jhz\" (UID: \"7e437f25-b6ba-42b8-b1c1-9245e8a86bc4\") " pod="openshift-dns/node-resolver-s2jhz" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710550 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9tzp\" (UniqueName: \"kubernetes.io/projected/7e437f25-b6ba-42b8-b1c1-9245e8a86bc4-kube-api-access-d9tzp\") pod \"node-resolver-s2jhz\" (UID: \"7e437f25-b6ba-42b8-b1c1-9245e8a86bc4\") " pod="openshift-dns/node-resolver-s2jhz" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710579 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e333f58-159f-4604-aefa-55d3aeb458b6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710612 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-system-cni-dir\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710640 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-multus-cni-dir\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710664 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b5208c81-86ee-4635-9dd3-137393805762-dbus\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710724 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e333f58-159f-4604-aefa-55d3aeb458b6-os-release\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710750 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5e23e056-ad6d-42d0-a5b2-54ebc60e3a9c-agent-certs\") pod \"konnectivity-agent-b7h24\" (UID: \"5e23e056-ad6d-42d0-a5b2-54ebc60e3a9c\") " pod="kube-system/konnectivity-agent-b7h24" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710780 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-multus-socket-dir-parent\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.710867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710805 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-var-lib-cni-multus\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.711766 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710837 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e333f58-159f-4604-aefa-55d3aeb458b6-system-cni-dir\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.711766 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710868 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-run-multus-certs\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.711766 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710875 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2bq4s\"" Jul 28 19:36:46.711766 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.710940 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 28 19:36:46.711766 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.711378 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 28 19:36:46.711766 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.711684 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 28 19:36:46.711766 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.711701 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r6cqm\"" Jul 28 19:36:46.712084 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.711955 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-n8d7f\"" Jul 28 19:36:46.712271 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.712251 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 28 19:36:46.712344 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.712307 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 28 19:36:46.712397 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.712383 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 28 19:36:46.712571 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.712551 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dbm9w\"" Jul 28 19:36:46.712638 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.712612 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 28 19:36:46.714208 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.714186 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 28 19:36:46.714482 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.714459 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 28 19:36:46.714692 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.714675 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 28 19:36:46.714991 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.714962 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 28 19:36:46.715556 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.715534 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 28 19:36:46.715848 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.715829 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6tgck\"" Jul 28 19:36:46.716282 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.716262 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 28 19:36:46.717415 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.717393 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 19:36:46.740381 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.740362 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pmjd2" Jul 28 19:36:46.752241 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.752209 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pmjd2" Jul 28 19:36:46.798068 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.798044 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 28 19:36:46.811111 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811084 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1be64912-89f6-4b48-a5d2-9a3aac8acdcf-host-slash\") pod \"iptables-alerter-s7ftg\" (UID: \"1be64912-89f6-4b48-a5d2-9a3aac8acdcf\") " pod="openshift-network-operator/iptables-alerter-s7ftg" Jul 28 19:36:46.811202 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811132 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5e23e056-ad6d-42d0-a5b2-54ebc60e3a9c-konnectivity-ca\") pod \"konnectivity-agent-b7h24\" (UID: \"5e23e056-ad6d-42d0-a5b2-54ebc60e3a9c\") " pod="kube-system/konnectivity-agent-b7h24" Jul 28 19:36:46.811202 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811161 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-run-netns\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.811202 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811188 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs\") pod \"network-metrics-daemon-q6qbg\" (UID: \"842143d0-262e-4779-90aa-50bb8b33633b\") " pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:46.811321 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811213 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/183d967b-aba7-4b25-843a-1de6360c3ad9-ovnkube-script-lib\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.811321 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811240 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.811321 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811266 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56jls\" (UniqueName: \"kubernetes.io/projected/1be64912-89f6-4b48-a5d2-9a3aac8acdcf-kube-api-access-56jls\") pod \"iptables-alerter-s7ftg\" (UID: \"1be64912-89f6-4b48-a5d2-9a3aac8acdcf\") " pod="openshift-network-operator/iptables-alerter-s7ftg" Jul 28 19:36:46.811321 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811290 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/095953aa-1212-406f-b21e-7b649915699d-host\") pod \"node-ca-6kkrr\" (UID: \"095953aa-1212-406f-b21e-7b649915699d\") " pod="openshift-image-registry/node-ca-6kkrr" Jul 28 19:36:46.811321 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811315 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-os-release\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.811496 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811341 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-hostroot\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.811496 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811366 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9e333f58-159f-4604-aefa-55d3aeb458b6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.811496 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811391 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-cnibin\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.811496 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811413 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-log-socket\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.811496 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811438 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-run-k8s-cni-cncf-io\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.811496 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811468 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e437f25-b6ba-42b8-b1c1-9245e8a86bc4-hosts-file\") pod \"node-resolver-s2jhz\" (UID: \"7e437f25-b6ba-42b8-b1c1-9245e8a86bc4\") " pod="openshift-dns/node-resolver-s2jhz" Jul 28 19:36:46.811496 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811494 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/183d967b-aba7-4b25-843a-1de6360c3ad9-ovn-node-metrics-cert\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.811795 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811519 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b5208c81-86ee-4635-9dd3-137393805762-kubelet-config\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:46.811795 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811546 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9e333f58-159f-4604-aefa-55d3aeb458b6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.811795 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811573 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-cni-binary-copy\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.811795 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811599 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-etc-kubernetes\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.811795 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811623 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-kubernetes\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.811795 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811646 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-slash\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.811795 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811670 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-etc-selinux\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.811795 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811724 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnz62\" (UniqueName: \"kubernetes.io/projected/6caf88fe-b980-421f-bfed-83cf7c76c49a-kube-api-access-rnz62\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.811795 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811753 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdvgn\" (UniqueName: \"kubernetes.io/projected/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-kube-api-access-mdvgn\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.811795 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811786 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9tzp\" (UniqueName: \"kubernetes.io/projected/7e437f25-b6ba-42b8-b1c1-9245e8a86bc4-kube-api-access-d9tzp\") pod \"node-resolver-s2jhz\" (UID: \"7e437f25-b6ba-42b8-b1c1-9245e8a86bc4\") " pod="openshift-dns/node-resolver-s2jhz" Jul 28 19:36:46.812226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811811 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-modprobe-d\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.812226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811844 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-run\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.812226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811868 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-systemd-units\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.812226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811892 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1be64912-89f6-4b48-a5d2-9a3aac8acdcf-iptables-alerter-script\") pod \"iptables-alerter-s7ftg\" (UID: \"1be64912-89f6-4b48-a5d2-9a3aac8acdcf\") " pod="openshift-network-operator/iptables-alerter-s7ftg" Jul 28 19:36:46.812226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811920 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-sys\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.812226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811947 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-host\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.812226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811971 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/183d967b-aba7-4b25-843a-1de6360c3ad9-env-overrides\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.812226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.811995 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b5208c81-86ee-4635-9dd3-137393805762-dbus\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:46.812226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.812021 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-multus-socket-dir-parent\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.812226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.812048 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-sysctl-conf\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.812226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.812086 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-tuned\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.812226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.812116 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-run-systemd\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.812226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.812131 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b5208c81-86ee-4635-9dd3-137393805762-kubelet-config\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:46.812672 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.812356 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b5208c81-86ee-4635-9dd3-137393805762-dbus\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:46.812672 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.812416 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-multus-socket-dir-parent\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.812800 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.812777 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5e23e056-ad6d-42d0-a5b2-54ebc60e3a9c-konnectivity-ca\") pod \"konnectivity-agent-b7h24\" (UID: \"5e23e056-ad6d-42d0-a5b2-54ebc60e3a9c\") " pod="kube-system/konnectivity-agent-b7h24" Jul 28 19:36:46.812847 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.812777 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9e333f58-159f-4604-aefa-55d3aeb458b6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.812882 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.812143 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-kubelet\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.812949 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.812928 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/095953aa-1212-406f-b21e-7b649915699d-serviceca\") pod \"node-ca-6kkrr\" (UID: \"095953aa-1212-406f-b21e-7b649915699d\") " pod="openshift-image-registry/node-ca-6kkrr" Jul 28 19:36:46.813000 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.812954 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-cnibin\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.813000 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.812973 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e333f58-159f-4604-aefa-55d3aeb458b6-cnibin\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.813078 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813005 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-multus-conf-dir\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.813078 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813021 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-cni-binary-copy\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.813078 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813030 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-multus-daemon-config\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.813169 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813078 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdwfh\" (UniqueName: \"kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh\") pod \"network-check-target-p4mcm\" (UID: \"7442e5a0-6ae1-43a9-b67e-ada4e63d3a41\") " pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:36:46.813169 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813113 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-run-openvswitch\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.813230 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813144 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-lib-modules\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.813230 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813195 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2mdr\" (UniqueName: \"kubernetes.io/projected/095953aa-1212-406f-b21e-7b649915699d-kube-api-access-d2mdr\") pod \"node-ca-6kkrr\" (UID: \"095953aa-1212-406f-b21e-7b649915699d\") " pod="openshift-image-registry/node-ca-6kkrr" Jul 28 19:36:46.813230 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813220 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-etc-kubernetes\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.813324 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813256 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e437f25-b6ba-42b8-b1c1-9245e8a86bc4-hosts-file\") pod \"node-resolver-s2jhz\" (UID: \"7e437f25-b6ba-42b8-b1c1-9245e8a86bc4\") " pod="openshift-dns/node-resolver-s2jhz" Jul 28 19:36:46.813324 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813310 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-run-netns\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.813324 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:46.813314 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 19:36:46.813415 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813352 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh72v\" (UniqueName: \"kubernetes.io/projected/9e333f58-159f-4604-aefa-55d3aeb458b6-kube-api-access-wh72v\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.813450 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:46.813413 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs podName:842143d0-262e-4779-90aa-50bb8b33633b nodeName:}" failed. No retries permitted until 2026-07-28 19:36:47.313375399 +0000 UTC m=+3.020587292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs") pod "network-metrics-daemon-q6qbg" (UID: "842143d0-262e-4779-90aa-50bb8b33633b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 19:36:46.813450 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813437 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:46.813450 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813416 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9e333f58-159f-4604-aefa-55d3aeb458b6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.813549 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813486 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-sysconfig\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.813549 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813512 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-systemd\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.813549 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813518 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-multus-daemon-config\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.813549 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813539 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-run-ovn\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.813680 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813567 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-hostroot\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.813680 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813621 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e333f58-159f-4604-aefa-55d3aeb458b6-cnibin\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.813680 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813641 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-os-release\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.813680 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813666 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-multus-conf-dir\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.813813 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813702 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-run-ovn-kubernetes\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.813813 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813717 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-run-k8s-cni-cncf-io\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.813813 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813766 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e333f58-159f-4604-aefa-55d3aeb458b6-cni-binary-copy\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.813813 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:46.813780 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 19:36:46.813813 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813804 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9hdh\" (UniqueName: \"kubernetes.io/projected/842143d0-262e-4779-90aa-50bb8b33633b-kube-api-access-r9hdh\") pod \"network-metrics-daemon-q6qbg\" (UID: \"842143d0-262e-4779-90aa-50bb8b33633b\") " pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:46.813962 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813835 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-var-lib-kubelet\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.813962 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:46.813844 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret podName:b5208c81-86ee-4635-9dd3-137393805762 nodeName:}" failed. No retries permitted until 2026-07-28 19:36:47.313827361 +0000 UTC m=+3.021039234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret") pod "global-pull-secret-syncer-2xb59" (UID: "b5208c81-86ee-4635-9dd3-137393805762") : object "kube-system"/"original-pull-secret" not registered Jul 28 19:36:46.813962 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813886 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d0b2fb-c3e8-47d8-acc1-08209f145787-tmp\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.813962 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.813918 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-run-netns\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.814100 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814083 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-cni-bin\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.814138 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814110 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/183d967b-aba7-4b25-843a-1de6360c3ad9-ovnkube-config\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.814138 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814129 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-socket-dir\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.814217 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814159 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-var-lib-cni-bin\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.814217 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814179 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-var-lib-openvswitch\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.814217 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814195 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-etc-openvswitch\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.814311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814214 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-node-log\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.814311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814249 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-var-lib-kubelet\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.814311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814258 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-var-lib-cni-bin\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.814311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814288 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-var-lib-kubelet\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.814311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814292 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e333f58-159f-4604-aefa-55d3aeb458b6-cni-binary-copy\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.814311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814304 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e437f25-b6ba-42b8-b1c1-9245e8a86bc4-tmp-dir\") pod \"node-resolver-s2jhz\" (UID: \"7e437f25-b6ba-42b8-b1c1-9245e8a86bc4\") " pod="openshift-dns/node-resolver-s2jhz" Jul 28 19:36:46.814529 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814335 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-sysctl-d\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.814529 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814437 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-registration-dir\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.814529 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814468 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-sys-fs\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.814529 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814497 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e333f58-159f-4604-aefa-55d3aeb458b6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.814529 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814524 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-system-cni-dir\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.814723 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814549 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-multus-cni-dir\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.814723 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814576 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4rwh\" (UniqueName: \"kubernetes.io/projected/00d0b2fb-c3e8-47d8-acc1-08209f145787-kube-api-access-l4rwh\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.814723 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814601 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-device-dir\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.814723 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814627 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e333f58-159f-4604-aefa-55d3aeb458b6-os-release\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.814723 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814629 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-system-cni-dir\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.814723 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814616 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e437f25-b6ba-42b8-b1c1-9245e8a86bc4-tmp-dir\") pod \"node-resolver-s2jhz\" (UID: \"7e437f25-b6ba-42b8-b1c1-9245e8a86bc4\") " pod="openshift-dns/node-resolver-s2jhz" Jul 28 19:36:46.814723 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814670 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5e23e056-ad6d-42d0-a5b2-54ebc60e3a9c-agent-certs\") pod \"konnectivity-agent-b7h24\" (UID: \"5e23e056-ad6d-42d0-a5b2-54ebc60e3a9c\") " pod="kube-system/konnectivity-agent-b7h24" Jul 28 19:36:46.814723 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814700 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-var-lib-cni-multus\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.814723 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814720 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-multus-cni-dir\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.815112 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814729 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.815112 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814760 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e333f58-159f-4604-aefa-55d3aeb458b6-system-cni-dir\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.815112 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814753 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e333f58-159f-4604-aefa-55d3aeb458b6-os-release\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.815112 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814794 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-run-multus-certs\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.815112 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814834 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-var-lib-cni-multus\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.815112 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814845 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-host-run-multus-certs\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.815112 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814869 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e333f58-159f-4604-aefa-55d3aeb458b6-system-cni-dir\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.815112 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814907 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-cni-netd\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.815112 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.814942 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78h2b\" (UniqueName: \"kubernetes.io/projected/183d967b-aba7-4b25-843a-1de6360c3ad9-kube-api-access-78h2b\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.815112 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.815054 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 28 19:36:46.815455 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.815116 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e333f58-159f-4604-aefa-55d3aeb458b6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.820436 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.820412 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5e23e056-ad6d-42d0-a5b2-54ebc60e3a9c-agent-certs\") pod \"konnectivity-agent-b7h24\" (UID: \"5e23e056-ad6d-42d0-a5b2-54ebc60e3a9c\") " pod="kube-system/konnectivity-agent-b7h24" Jul 28 19:36:46.828774 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:46.828754 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 19:36:46.828881 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:46.828780 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 19:36:46.828881 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:46.828795 2564 projected.go:196] Error preparing data for projected volume kube-api-access-mdwfh for pod openshift-network-diagnostics/network-check-target-p4mcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 19:36:46.828881 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:46.828853 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh podName:7442e5a0-6ae1-43a9-b67e-ada4e63d3a41 nodeName:}" failed. No retries permitted until 2026-07-28 19:36:47.328835823 +0000 UTC m=+3.036047696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mdwfh" (UniqueName: "kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh") pod "network-check-target-p4mcm" (UID: "7442e5a0-6ae1-43a9-b67e-ada4e63d3a41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 19:36:46.832493 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.832470 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9tzp\" (UniqueName: \"kubernetes.io/projected/7e437f25-b6ba-42b8-b1c1-9245e8a86bc4-kube-api-access-d9tzp\") pod \"node-resolver-s2jhz\" (UID: \"7e437f25-b6ba-42b8-b1c1-9245e8a86bc4\") " pod="openshift-dns/node-resolver-s2jhz" Jul 28 19:36:46.833148 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.833125 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9hdh\" (UniqueName: \"kubernetes.io/projected/842143d0-262e-4779-90aa-50bb8b33633b-kube-api-access-r9hdh\") pod \"network-metrics-daemon-q6qbg\" (UID: \"842143d0-262e-4779-90aa-50bb8b33633b\") " pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:46.834708 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.834684 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh72v\" (UniqueName: \"kubernetes.io/projected/9e333f58-159f-4604-aefa-55d3aeb458b6-kube-api-access-wh72v\") pod \"multus-additional-cni-plugins-gpbk6\" (UID: \"9e333f58-159f-4604-aefa-55d3aeb458b6\") " pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:46.836921 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.836886 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdvgn\" (UniqueName: \"kubernetes.io/projected/aefbbe19-d86c-40e6-a2c0-dbfded5e17b9-kube-api-access-mdvgn\") pod \"multus-tj7qj\" (UID: \"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9\") " pod="openshift-multus/multus-tj7qj" Jul 28 19:36:46.915285 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915251 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-sysconfig\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.915285 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915281 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-systemd\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.915530 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915303 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-run-ovn\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.915530 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915325 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-run-ovn-kubernetes\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.915530 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915342 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-var-lib-kubelet\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.915530 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915357 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d0b2fb-c3e8-47d8-acc1-08209f145787-tmp\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.915530 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915371 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-run-netns\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.915530 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915386 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-cni-bin\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.915530 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915409 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/183d967b-aba7-4b25-843a-1de6360c3ad9-ovnkube-config\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.915530 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915430 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-socket-dir\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.915530 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915457 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-var-lib-openvswitch\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.915530 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915472 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-etc-openvswitch\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.915530 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915502 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-node-log\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.915530 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915525 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-sysctl-d\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915544 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-registration-dir\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915560 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-sys-fs\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915582 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4rwh\" (UniqueName: \"kubernetes.io/projected/00d0b2fb-c3e8-47d8-acc1-08209f145787-kube-api-access-l4rwh\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915598 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-device-dir\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915624 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915652 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-cni-netd\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915696 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78h2b\" (UniqueName: \"kubernetes.io/projected/183d967b-aba7-4b25-843a-1de6360c3ad9-kube-api-access-78h2b\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915733 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-run-ovn-kubernetes\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915743 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1be64912-89f6-4b48-a5d2-9a3aac8acdcf-host-slash\") pod \"iptables-alerter-s7ftg\" (UID: \"1be64912-89f6-4b48-a5d2-9a3aac8acdcf\") " pod="openshift-network-operator/iptables-alerter-s7ftg" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915802 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/183d967b-aba7-4b25-843a-1de6360c3ad9-ovnkube-script-lib\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915806 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1be64912-89f6-4b48-a5d2-9a3aac8acdcf-host-slash\") pod \"iptables-alerter-s7ftg\" (UID: \"1be64912-89f6-4b48-a5d2-9a3aac8acdcf\") " pod="openshift-network-operator/iptables-alerter-s7ftg" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915834 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915861 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56jls\" (UniqueName: \"kubernetes.io/projected/1be64912-89f6-4b48-a5d2-9a3aac8acdcf-kube-api-access-56jls\") pod \"iptables-alerter-s7ftg\" (UID: \"1be64912-89f6-4b48-a5d2-9a3aac8acdcf\") " pod="openshift-network-operator/iptables-alerter-s7ftg" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915885 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/095953aa-1212-406f-b21e-7b649915699d-host\") pod \"node-ca-6kkrr\" (UID: \"095953aa-1212-406f-b21e-7b649915699d\") " pod="openshift-image-registry/node-ca-6kkrr" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915915 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-log-socket\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915944 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/183d967b-aba7-4b25-843a-1de6360c3ad9-ovn-node-metrics-cert\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.916160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915976 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-kubernetes\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915983 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-var-lib-openvswitch\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916005 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-slash\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916041 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-etc-openvswitch\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916076 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-slash\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916132 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-sysconfig\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916177 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-systemd\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916191 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-node-log\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916215 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-run-ovn\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916303 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/183d967b-aba7-4b25-843a-1de6360c3ad9-ovnkube-config\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916336 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-sysctl-d\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916386 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-var-lib-kubelet\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916440 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-registration-dir\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916517 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-sys-fs\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916846 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/183d967b-aba7-4b25-843a-1de6360c3ad9-ovnkube-script-lib\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916846 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-device-dir\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916923 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916923 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-cni-netd\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.917009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.916971 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/095953aa-1212-406f-b21e-7b649915699d-host\") pod \"node-ca-6kkrr\" (UID: \"095953aa-1212-406f-b21e-7b649915699d\") " pod="openshift-image-registry/node-ca-6kkrr" Jul 28 19:36:46.917764 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917013 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-log-socket\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.917764 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917148 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-etc-selinux\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.917764 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917181 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnz62\" (UniqueName: \"kubernetes.io/projected/6caf88fe-b980-421f-bfed-83cf7c76c49a-kube-api-access-rnz62\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.917764 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917191 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-run-netns\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.917764 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917214 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-modprobe-d\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.917764 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917260 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-run\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.917764 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917354 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-cni-bin\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.917764 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917412 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-systemd-units\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.917764 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917448 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1be64912-89f6-4b48-a5d2-9a3aac8acdcf-iptables-alerter-script\") pod \"iptables-alerter-s7ftg\" (UID: \"1be64912-89f6-4b48-a5d2-9a3aac8acdcf\") " pod="openshift-network-operator/iptables-alerter-s7ftg" Jul 28 19:36:46.917764 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917482 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-sys\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.917764 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917506 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-host\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.917764 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917558 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/183d967b-aba7-4b25-843a-1de6360c3ad9-env-overrides\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.917764 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917730 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-sysctl-conf\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.918225 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917768 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-tuned\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.918225 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917802 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-run-systemd\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.918225 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917850 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-kubelet\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.918225 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917875 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/095953aa-1212-406f-b21e-7b649915699d-serviceca\") pod \"node-ca-6kkrr\" (UID: \"095953aa-1212-406f-b21e-7b649915699d\") " pod="openshift-image-registry/node-ca-6kkrr" Jul 28 19:36:46.918225 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917931 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-run-openvswitch\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.918225 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917959 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-lib-modules\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.918225 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.917992 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2mdr\" (UniqueName: \"kubernetes.io/projected/095953aa-1212-406f-b21e-7b649915699d-kube-api-access-d2mdr\") pod \"node-ca-6kkrr\" (UID: \"095953aa-1212-406f-b21e-7b649915699d\") " pod="openshift-image-registry/node-ca-6kkrr" Jul 28 19:36:46.918451 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.918378 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-kubernetes\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.918451 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.915945 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-socket-dir\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.918671 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.918637 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-sys\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.918779 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.918713 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-run\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.918912 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.918874 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/183d967b-aba7-4b25-843a-1de6360c3ad9-env-overrides\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.918984 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.918967 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-host\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.919047 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.918966 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-modprobe-d\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.919047 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.919004 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-systemd-units\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.919148 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.919044 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6caf88fe-b980-421f-bfed-83cf7c76c49a-etc-selinux\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.919148 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.919067 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.919237 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.919141 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-sysctl-conf\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.919295 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.919233 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-run-openvswitch\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.919351 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.919304 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/00d0b2fb-c3e8-47d8-acc1-08209f145787-lib-modules\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.919406 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.919350 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d0b2fb-c3e8-47d8-acc1-08209f145787-tmp\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.920062 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.919823 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/183d967b-aba7-4b25-843a-1de6360c3ad9-ovn-node-metrics-cert\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.920062 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.919895 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-host-kubelet\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.920062 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.919940 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/183d967b-aba7-4b25-843a-1de6360c3ad9-run-systemd\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.920484 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.920454 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1be64912-89f6-4b48-a5d2-9a3aac8acdcf-iptables-alerter-script\") pod \"iptables-alerter-s7ftg\" (UID: \"1be64912-89f6-4b48-a5d2-9a3aac8acdcf\") " pod="openshift-network-operator/iptables-alerter-s7ftg" Jul 28 19:36:46.920886 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.920864 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/095953aa-1212-406f-b21e-7b649915699d-serviceca\") pod \"node-ca-6kkrr\" (UID: \"095953aa-1212-406f-b21e-7b649915699d\") " pod="openshift-image-registry/node-ca-6kkrr" Jul 28 19:36:46.921882 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.921857 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 19:36:46.922066 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.922038 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" event={"ID":"1eb1ec1d2f5d49622c78b5890bf891f2","Type":"ContainerStarted","Data":"7bee2b64bdc8b3e1840c7848d303a89fb9a06644f8338e3f5eeff3784d3596cf"} Jul 28 19:36:46.922219 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.922197 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/00d0b2fb-c3e8-47d8-acc1-08209f145787-etc-tuned\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.930899 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.930527 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnz62\" (UniqueName: \"kubernetes.io/projected/6caf88fe-b980-421f-bfed-83cf7c76c49a-kube-api-access-rnz62\") pod \"aws-ebs-csi-driver-node-q5hp4\" (UID: \"6caf88fe-b980-421f-bfed-83cf7c76c49a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:46.931194 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.931165 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2mdr\" (UniqueName: \"kubernetes.io/projected/095953aa-1212-406f-b21e-7b649915699d-kube-api-access-d2mdr\") pod \"node-ca-6kkrr\" (UID: \"095953aa-1212-406f-b21e-7b649915699d\") " pod="openshift-image-registry/node-ca-6kkrr" Jul 28 19:36:46.931797 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.931763 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4rwh\" (UniqueName: \"kubernetes.io/projected/00d0b2fb-c3e8-47d8-acc1-08209f145787-kube-api-access-l4rwh\") pod \"tuned-mq2zb\" (UID: \"00d0b2fb-c3e8-47d8-acc1-08209f145787\") " pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:46.932283 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.932254 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78h2b\" (UniqueName: \"kubernetes.io/projected/183d967b-aba7-4b25-843a-1de6360c3ad9-kube-api-access-78h2b\") pod \"ovnkube-node-vl4ft\" (UID: \"183d967b-aba7-4b25-843a-1de6360c3ad9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:46.933029 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:46.932986 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56jls\" (UniqueName: \"kubernetes.io/projected/1be64912-89f6-4b48-a5d2-9a3aac8acdcf-kube-api-access-56jls\") pod \"iptables-alerter-s7ftg\" (UID: \"1be64912-89f6-4b48-a5d2-9a3aac8acdcf\") " pod="openshift-network-operator/iptables-alerter-s7ftg" Jul 28 19:36:47.018080 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:47.017989 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b7h24" Jul 28 19:36:47.024795 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:47.024769 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gpbk6" Jul 28 19:36:47.037783 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:47.037759 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tj7qj" Jul 28 19:36:47.044578 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:47.044559 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s2jhz" Jul 28 19:36:47.053358 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:47.053336 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" Jul 28 19:36:47.066104 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:47.066086 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6kkrr" Jul 28 19:36:47.073824 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:47.073803 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" Jul 28 19:36:47.082618 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:47.082598 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s7ftg" Jul 28 19:36:47.094487 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:47.094467 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:36:47.320539 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:47.320446 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs\") pod \"network-metrics-daemon-q6qbg\" (UID: \"842143d0-262e-4779-90aa-50bb8b33633b\") " pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:47.320539 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:47.320531 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:47.320758 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:47.320663 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 19:36:47.320758 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:47.320741 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret podName:b5208c81-86ee-4635-9dd3-137393805762 nodeName:}" failed. No retries permitted until 2026-07-28 19:36:48.320727089 +0000 UTC m=+4.027938965 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret") pod "global-pull-secret-syncer-2xb59" (UID: "b5208c81-86ee-4635-9dd3-137393805762") : object "kube-system"/"original-pull-secret" not registered Jul 28 19:36:47.320875 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:47.320667 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 19:36:47.320875 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:47.320852 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs podName:842143d0-262e-4779-90aa-50bb8b33633b nodeName:}" failed. No retries permitted until 2026-07-28 19:36:48.320831159 +0000 UTC m=+4.028043037 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs") pod "network-metrics-daemon-q6qbg" (UID: "842143d0-262e-4779-90aa-50bb8b33633b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 19:36:47.421544 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:47.421508 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdwfh\" (UniqueName: \"kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh\") pod \"network-check-target-p4mcm\" (UID: \"7442e5a0-6ae1-43a9-b67e-ada4e63d3a41\") " pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:36:47.421752 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:47.421723 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 19:36:47.421752 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:47.421749 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 19:36:47.421873 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:47.421763 2564 projected.go:196] Error preparing data for projected volume kube-api-access-mdwfh for pod openshift-network-diagnostics/network-check-target-p4mcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 19:36:47.421873 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:47.421836 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh podName:7442e5a0-6ae1-43a9-b67e-ada4e63d3a41 nodeName:}" failed. No retries permitted until 2026-07-28 19:36:48.421814587 +0000 UTC m=+4.129026479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mdwfh" (UniqueName: "kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh") pod "network-check-target-p4mcm" (UID: "7442e5a0-6ae1-43a9-b67e-ada4e63d3a41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 19:36:47.756277 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:47.756176 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 19:31:46 +0000 UTC" deadline="2027-12-30 22:27:54.129541692 +0000 UTC" Jul 28 19:36:47.756277 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:47.756210 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12482h51m6.373335591s" Jul 28 19:36:47.912627 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:47.912592 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:47.912831 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:47.912748 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:36:48.009606 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:48.009576 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00d0b2fb_c3e8_47d8_acc1_08209f145787.slice/crio-d1d5dd170e25edfd36729a2b728e0e60b3420d0ed87b6f777b1602994ea4ea9f WatchSource:0}: Error finding container d1d5dd170e25edfd36729a2b728e0e60b3420d0ed87b6f777b1602994ea4ea9f: Status 404 returned error can't find the container with id d1d5dd170e25edfd36729a2b728e0e60b3420d0ed87b6f777b1602994ea4ea9f Jul 28 19:36:48.017254 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:48.017230 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod095953aa_1212_406f_b21e_7b649915699d.slice/crio-cfc03e163869f52773f36e979b6ecdd95015fefa3d55d52939e8b73a980d29a1 WatchSource:0}: Error finding container cfc03e163869f52773f36e979b6ecdd95015fefa3d55d52939e8b73a980d29a1: Status 404 returned error can't find the container with id cfc03e163869f52773f36e979b6ecdd95015fefa3d55d52939e8b73a980d29a1 Jul 28 19:36:48.019268 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:48.019224 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaefbbe19_d86c_40e6_a2c0_dbfded5e17b9.slice/crio-35aa1a5ced0715734bb846a53cd044229ed5726471bfe619b275c27e9913bf66 WatchSource:0}: Error finding container 35aa1a5ced0715734bb846a53cd044229ed5726471bfe619b275c27e9913bf66: Status 404 returned error can't find the container with id 35aa1a5ced0715734bb846a53cd044229ed5726471bfe619b275c27e9913bf66 Jul 28 19:36:48.020420 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:48.020401 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e437f25_b6ba_42b8_b1c1_9245e8a86bc4.slice/crio-54740882dbaa8659e5c93fb59978886dd8768431dd76369c24913ee85ec59c80 WatchSource:0}: Error finding container 54740882dbaa8659e5c93fb59978886dd8768431dd76369c24913ee85ec59c80: Status 404 returned error can't find the container with id 54740882dbaa8659e5c93fb59978886dd8768431dd76369c24913ee85ec59c80 Jul 28 19:36:48.022794 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:48.022760 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1be64912_89f6_4b48_a5d2_9a3aac8acdcf.slice/crio-14c191d5ec3e41ee5c94f9af18d031538854252fde0fbe094ccbfab4090d825c WatchSource:0}: Error finding container 14c191d5ec3e41ee5c94f9af18d031538854252fde0fbe094ccbfab4090d825c: Status 404 returned error can't find the container with id 14c191d5ec3e41ee5c94f9af18d031538854252fde0fbe094ccbfab4090d825c Jul 28 19:36:48.023731 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:48.023684 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e23e056_ad6d_42d0_a5b2_54ebc60e3a9c.slice/crio-8e8367eefdbaa394bde8157185d69f94ea01197dc2ab471e395c4e48bd5d8ba2 WatchSource:0}: Error finding container 8e8367eefdbaa394bde8157185d69f94ea01197dc2ab471e395c4e48bd5d8ba2: Status 404 returned error can't find the container with id 8e8367eefdbaa394bde8157185d69f94ea01197dc2ab471e395c4e48bd5d8ba2 Jul 28 19:36:48.025409 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:48.025384 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod183d967b_aba7_4b25_843a_1de6360c3ad9.slice/crio-57e2d85187167572cd485644abc6802590f92fc78ed867a908cd9314b6ef3976 WatchSource:0}: Error finding container 57e2d85187167572cd485644abc6802590f92fc78ed867a908cd9314b6ef3976: Status 404 returned error can't find the container with id 57e2d85187167572cd485644abc6802590f92fc78ed867a908cd9314b6ef3976 Jul 28 19:36:48.027882 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:36:48.027856 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e333f58_159f_4604_aefa_55d3aeb458b6.slice/crio-8806313a200c50544ef7494543cdf89d856eeb0c6972474ba06e31651566aa95 WatchSource:0}: Error finding container 8806313a200c50544ef7494543cdf89d856eeb0c6972474ba06e31651566aa95: Status 404 returned error can't find the container with id 8806313a200c50544ef7494543cdf89d856eeb0c6972474ba06e31651566aa95 Jul 28 19:36:48.327058 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:48.327018 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs\") pod \"network-metrics-daemon-q6qbg\" (UID: \"842143d0-262e-4779-90aa-50bb8b33633b\") " pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:48.327245 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:48.327095 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:48.327245 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:48.327217 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 19:36:48.327340 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:48.327243 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 19:36:48.327340 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:48.327292 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs podName:842143d0-262e-4779-90aa-50bb8b33633b nodeName:}" failed. No retries permitted until 2026-07-28 19:36:50.327273063 +0000 UTC m=+6.034484957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs") pod "network-metrics-daemon-q6qbg" (UID: "842143d0-262e-4779-90aa-50bb8b33633b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 19:36:48.327340 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:48.327309 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret podName:b5208c81-86ee-4635-9dd3-137393805762 nodeName:}" failed. No retries permitted until 2026-07-28 19:36:50.327301756 +0000 UTC m=+6.034513632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret") pod "global-pull-secret-syncer-2xb59" (UID: "b5208c81-86ee-4635-9dd3-137393805762") : object "kube-system"/"original-pull-secret" not registered Jul 28 19:36:48.427987 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:48.427940 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdwfh\" (UniqueName: \"kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh\") pod \"network-check-target-p4mcm\" (UID: \"7442e5a0-6ae1-43a9-b67e-ada4e63d3a41\") " pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:36:48.428134 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:48.428105 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 19:36:48.428134 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:48.428123 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 19:36:48.428134 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:48.428133 2564 projected.go:196] Error preparing data for projected volume kube-api-access-mdwfh for pod openshift-network-diagnostics/network-check-target-p4mcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 19:36:48.428262 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:48.428187 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh podName:7442e5a0-6ae1-43a9-b67e-ada4e63d3a41 nodeName:}" failed. No retries permitted until 2026-07-28 19:36:50.428169245 +0000 UTC m=+6.135381129 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mdwfh" (UniqueName: "kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh") pod "network-check-target-p4mcm" (UID: "7442e5a0-6ae1-43a9-b67e-ada4e63d3a41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 19:36:48.756949 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:48.756903 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 19:31:46 +0000 UTC" deadline="2027-12-30 00:12:39.739391812 +0000 UTC" Jul 28 19:36:48.756949 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:48.756948 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12460h35m50.982449263s" Jul 28 19:36:48.914388 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:48.913856 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:48.914388 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:48.914006 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:36:48.914388 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:48.914088 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:36:48.914388 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:48.914191 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:36:48.943011 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:48.942935 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tj7qj" event={"ID":"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9","Type":"ContainerStarted","Data":"35aa1a5ced0715734bb846a53cd044229ed5726471bfe619b275c27e9913bf66"} Jul 28 19:36:48.944782 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:48.944687 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6kkrr" event={"ID":"095953aa-1212-406f-b21e-7b649915699d","Type":"ContainerStarted","Data":"cfc03e163869f52773f36e979b6ecdd95015fefa3d55d52939e8b73a980d29a1"} Jul 28 19:36:48.949774 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:48.949380 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" event={"ID":"1eb1ec1d2f5d49622c78b5890bf891f2","Type":"ContainerStarted","Data":"b98125c2f67300428536384e3735af30d22076ee6cae4636e825f39584982d60"} Jul 28 19:36:48.959140 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:48.959042 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpbk6" event={"ID":"9e333f58-159f-4604-aefa-55d3aeb458b6","Type":"ContainerStarted","Data":"8806313a200c50544ef7494543cdf89d856eeb0c6972474ba06e31651566aa95"} Jul 28 19:36:48.966492 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:48.966179 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" podStartSLOduration=2.966161887 podStartE2EDuration="2.966161887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:36:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:48.965516057 +0000 UTC m=+4.672727955" watchObservedRunningTime="2026-07-28 19:36:48.966161887 +0000 UTC m=+4.673373782" Jul 28 19:36:48.970372 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:48.970314 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b7h24" event={"ID":"5e23e056-ad6d-42d0-a5b2-54ebc60e3a9c","Type":"ContainerStarted","Data":"8e8367eefdbaa394bde8157185d69f94ea01197dc2ab471e395c4e48bd5d8ba2"} Jul 28 19:36:48.980561 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:48.980525 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s7ftg" event={"ID":"1be64912-89f6-4b48-a5d2-9a3aac8acdcf","Type":"ContainerStarted","Data":"14c191d5ec3e41ee5c94f9af18d031538854252fde0fbe094ccbfab4090d825c"} Jul 28 19:36:48.994436 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:48.994315 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s2jhz" event={"ID":"7e437f25-b6ba-42b8-b1c1-9245e8a86bc4","Type":"ContainerStarted","Data":"54740882dbaa8659e5c93fb59978886dd8768431dd76369c24913ee85ec59c80"} Jul 28 19:36:49.000243 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:49.000206 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" event={"ID":"00d0b2fb-c3e8-47d8-acc1-08209f145787","Type":"ContainerStarted","Data":"d1d5dd170e25edfd36729a2b728e0e60b3420d0ed87b6f777b1602994ea4ea9f"} Jul 28 19:36:49.025115 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:49.025085 2564 generic.go:356] "Generic (PLEG): container finished" podID="06f51ddf0048f10e09fab733759c2c7c" containerID="5896f65f80713ea44c67489308d4865ee4675b6d455f4757cbb9f3484199b25f" exitCode=0 Jul 28 19:36:49.025242 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:49.025179 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" event={"ID":"06f51ddf0048f10e09fab733759c2c7c","Type":"ContainerDied","Data":"5896f65f80713ea44c67489308d4865ee4675b6d455f4757cbb9f3484199b25f"} Jul 28 19:36:49.030296 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:49.030229 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" event={"ID":"6caf88fe-b980-421f-bfed-83cf7c76c49a","Type":"ContainerStarted","Data":"48fd4d49691045b3824743f11abda16d1377c3d8522ffeac7a842c308a816bd0"} Jul 28 19:36:49.037249 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:49.037121 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" event={"ID":"183d967b-aba7-4b25-843a-1de6360c3ad9","Type":"ContainerStarted","Data":"57e2d85187167572cd485644abc6802590f92fc78ed867a908cd9314b6ef3976"} Jul 28 19:36:49.913317 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:49.913281 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:49.914126 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:49.913437 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:36:50.046489 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:50.046282 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" event={"ID":"06f51ddf0048f10e09fab733759c2c7c","Type":"ContainerStarted","Data":"d7fd985085f6981c0401720201088e64aefe21e9353605248a50ab6ea181d775"} Jul 28 19:36:50.344608 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:50.344520 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:50.344608 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:50.344596 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs\") pod \"network-metrics-daemon-q6qbg\" (UID: \"842143d0-262e-4779-90aa-50bb8b33633b\") " pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:50.344844 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:50.344736 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 19:36:50.344844 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:50.344798 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs podName:842143d0-262e-4779-90aa-50bb8b33633b nodeName:}" failed. No retries permitted until 2026-07-28 19:36:54.344780219 +0000 UTC m=+10.051992107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs") pod "network-metrics-daemon-q6qbg" (UID: "842143d0-262e-4779-90aa-50bb8b33633b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 19:36:50.344961 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:50.344869 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 19:36:50.344961 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:50.344909 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret podName:b5208c81-86ee-4635-9dd3-137393805762 nodeName:}" failed. No retries permitted until 2026-07-28 19:36:54.344897015 +0000 UTC m=+10.052108890 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret") pod "global-pull-secret-syncer-2xb59" (UID: "b5208c81-86ee-4635-9dd3-137393805762") : object "kube-system"/"original-pull-secret" not registered Jul 28 19:36:50.445090 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:50.445049 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdwfh\" (UniqueName: \"kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh\") pod \"network-check-target-p4mcm\" (UID: \"7442e5a0-6ae1-43a9-b67e-ada4e63d3a41\") " pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:36:50.445347 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:50.445283 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 19:36:50.445347 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:50.445310 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 19:36:50.445347 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:50.445322 2564 projected.go:196] Error preparing data for projected volume kube-api-access-mdwfh for pod openshift-network-diagnostics/network-check-target-p4mcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 19:36:50.445500 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:50.445383 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh podName:7442e5a0-6ae1-43a9-b67e-ada4e63d3a41 nodeName:}" failed. No retries permitted until 2026-07-28 19:36:54.445364551 +0000 UTC m=+10.152576441 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mdwfh" (UniqueName: "kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh") pod "network-check-target-p4mcm" (UID: "7442e5a0-6ae1-43a9-b67e-ada4e63d3a41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 19:36:50.912297 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:50.912265 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:50.912512 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:50.912390 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:36:50.912818 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:50.912788 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:36:50.912907 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:50.912878 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:36:51.912838 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:51.912802 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:51.913322 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:51.912953 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:36:52.912865 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:52.912784 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:36:52.913308 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:52.912924 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:36:52.913308 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:52.913287 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:52.913421 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:52.913375 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:36:53.913039 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:53.912998 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:53.913460 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:53.913156 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:36:54.375034 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:54.374942 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:54.375034 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:54.375011 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs\") pod \"network-metrics-daemon-q6qbg\" (UID: \"842143d0-262e-4779-90aa-50bb8b33633b\") " pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:54.375262 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:54.375149 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 19:36:54.375262 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:54.375212 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs podName:842143d0-262e-4779-90aa-50bb8b33633b nodeName:}" failed. No retries permitted until 2026-07-28 19:37:02.375194002 +0000 UTC m=+18.082405879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs") pod "network-metrics-daemon-q6qbg" (UID: "842143d0-262e-4779-90aa-50bb8b33633b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 19:36:54.375601 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:54.375582 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 19:36:54.375662 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:54.375651 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret podName:b5208c81-86ee-4635-9dd3-137393805762 nodeName:}" failed. No retries permitted until 2026-07-28 19:37:02.375621595 +0000 UTC m=+18.082833471 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret") pod "global-pull-secret-syncer-2xb59" (UID: "b5208c81-86ee-4635-9dd3-137393805762") : object "kube-system"/"original-pull-secret" not registered Jul 28 19:36:54.475782 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:54.475659 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdwfh\" (UniqueName: \"kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh\") pod \"network-check-target-p4mcm\" (UID: \"7442e5a0-6ae1-43a9-b67e-ada4e63d3a41\") " pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:36:54.475988 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:54.475852 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 19:36:54.475988 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:54.475879 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 19:36:54.475988 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:54.475892 2564 projected.go:196] Error preparing data for projected volume kube-api-access-mdwfh for pod openshift-network-diagnostics/network-check-target-p4mcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 19:36:54.475988 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:54.475957 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh podName:7442e5a0-6ae1-43a9-b67e-ada4e63d3a41 nodeName:}" failed. No retries permitted until 2026-07-28 19:37:02.475938022 +0000 UTC m=+18.183149913 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mdwfh" (UniqueName: "kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh") pod "network-check-target-p4mcm" (UID: "7442e5a0-6ae1-43a9-b67e-ada4e63d3a41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 19:36:54.913984 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:54.913951 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:36:54.914503 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:54.914064 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:36:54.914503 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:54.914441 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:54.914706 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:54.914530 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:36:55.912796 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:55.912764 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:55.913018 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:55.912900 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:36:56.912754 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:56.912718 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:56.913178 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:56.912851 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:36:56.913178 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:56.913089 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:36:56.913178 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:56.913171 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:36:57.912690 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:57.912652 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:57.912880 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:57.912802 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:36:58.915940 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:58.915911 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:36:58.916397 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:58.915919 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:36:58.916397 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:58.916023 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:36:58.916397 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:58.916122 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:36:59.913128 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:36:59.913095 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:36:59.913319 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:36:59.913221 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:00.912883 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:00.912848 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:00.913321 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:00.912988 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:00.913321 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:00.913051 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:00.913321 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:00.913164 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:01.913157 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:01.913122 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:01.913615 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:01.913258 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:02.436931 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:02.436889 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:02.437088 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:02.436961 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs\") pod \"network-metrics-daemon-q6qbg\" (UID: \"842143d0-262e-4779-90aa-50bb8b33633b\") " pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:02.437088 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:02.437044 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 19:37:02.437088 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:02.437082 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 19:37:02.437209 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:02.437112 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret podName:b5208c81-86ee-4635-9dd3-137393805762 nodeName:}" failed. No retries permitted until 2026-07-28 19:37:18.437094202 +0000 UTC m=+34.144306089 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret") pod "global-pull-secret-syncer-2xb59" (UID: "b5208c81-86ee-4635-9dd3-137393805762") : object "kube-system"/"original-pull-secret" not registered Jul 28 19:37:02.437209 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:02.437134 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs podName:842143d0-262e-4779-90aa-50bb8b33633b nodeName:}" failed. No retries permitted until 2026-07-28 19:37:18.437122971 +0000 UTC m=+34.144334855 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs") pod "network-metrics-daemon-q6qbg" (UID: "842143d0-262e-4779-90aa-50bb8b33633b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 19:37:02.538247 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:02.538213 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdwfh\" (UniqueName: \"kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh\") pod \"network-check-target-p4mcm\" (UID: \"7442e5a0-6ae1-43a9-b67e-ada4e63d3a41\") " pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:02.538398 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:02.538364 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 19:37:02.538398 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:02.538382 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 19:37:02.538398 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:02.538392 2564 projected.go:196] Error preparing data for projected volume kube-api-access-mdwfh for pod openshift-network-diagnostics/network-check-target-p4mcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 19:37:02.538515 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:02.538438 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh podName:7442e5a0-6ae1-43a9-b67e-ada4e63d3a41 nodeName:}" failed. No retries permitted until 2026-07-28 19:37:18.538424885 +0000 UTC m=+34.245636760 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mdwfh" (UniqueName: "kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh") pod "network-check-target-p4mcm" (UID: "7442e5a0-6ae1-43a9-b67e-ada4e63d3a41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 19:37:02.913038 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:02.912957 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:02.913038 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:02.913002 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:02.913515 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:02.913088 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:02.913515 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:02.913206 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:03.912510 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:03.912474 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:03.912715 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:03.912606 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:04.914245 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:04.914205 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:04.914713 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:04.914330 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:04.914713 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:04.914385 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:04.914713 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:04.914501 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:05.912926 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:05.912888 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:05.913100 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:05.913026 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:06.913724 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:06.913213 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:06.914515 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:06.913722 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:06.914515 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:06.913841 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:06.914515 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:06.913950 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:07.084343 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.084260 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tj7qj" event={"ID":"aefbbe19-d86c-40e6-a2c0-dbfded5e17b9","Type":"ContainerStarted","Data":"5083adc0d9358abe4742d9408e3a163707152d9e50efe753e242f9f5a20a199b"} Jul 28 19:37:07.085906 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.085876 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6kkrr" event={"ID":"095953aa-1212-406f-b21e-7b649915699d","Type":"ContainerStarted","Data":"4de1fddb6ed12502e47580d5e08497862ea8772b97ff71fe011593a433bf3980"} Jul 28 19:37:07.087516 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.087489 2564 generic.go:356] "Generic (PLEG): container finished" podID="9e333f58-159f-4604-aefa-55d3aeb458b6" containerID="1e96b9bae9948c51973767403a22a0705cbebdceb09fb2c447e699727ed30aac" exitCode=0 Jul 28 19:37:07.087657 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.087518 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpbk6" event={"ID":"9e333f58-159f-4604-aefa-55d3aeb458b6","Type":"ContainerDied","Data":"1e96b9bae9948c51973767403a22a0705cbebdceb09fb2c447e699727ed30aac"} Jul 28 19:37:07.089212 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.089158 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b7h24" event={"ID":"5e23e056-ad6d-42d0-a5b2-54ebc60e3a9c","Type":"ContainerStarted","Data":"f2affc4fbc8269ba26e18e7a001dd9b24f7e9c97e561024ccc798487e39be979"} Jul 28 19:37:07.090747 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.090724 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s2jhz" event={"ID":"7e437f25-b6ba-42b8-b1c1-9245e8a86bc4","Type":"ContainerStarted","Data":"c2a0b77dcf8664c13a11a05401eb91c887565d1d8903bda980889071a9435dfd"} Jul 28 19:37:07.092464 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.092439 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" event={"ID":"00d0b2fb-c3e8-47d8-acc1-08209f145787","Type":"ContainerStarted","Data":"da991774570b1bf4a543733719cbc7a3db8d67bd6fe5d1446145840454c3892c"} Jul 28 19:37:07.094148 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.094117 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" event={"ID":"6caf88fe-b980-421f-bfed-83cf7c76c49a","Type":"ContainerStarted","Data":"e8ee1670626815b794adc48d0f73c37fc1f5a71a2001fc421a52e68e102f2c9b"} Jul 28 19:37:07.097648 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.097613 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" event={"ID":"183d967b-aba7-4b25-843a-1de6360c3ad9","Type":"ContainerStarted","Data":"ecb9fd53d72d1e66f61666c2f875347a7ca77c3f21526dcede61dd371cc666c6"} Jul 28 19:37:07.097753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.097697 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" event={"ID":"183d967b-aba7-4b25-843a-1de6360c3ad9","Type":"ContainerStarted","Data":"519b744929d81699e5e1d5d445562af1004833ad8a2b708310a13d8233cac81d"} Jul 28 19:37:07.097831 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.097751 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" event={"ID":"183d967b-aba7-4b25-843a-1de6360c3ad9","Type":"ContainerStarted","Data":"d6c6a07ca3c0331ce17ef9da5268bab9ba0a2445dd22bb0a125974351d74c919"} Jul 28 19:37:07.097831 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.097806 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" event={"ID":"183d967b-aba7-4b25-843a-1de6360c3ad9","Type":"ContainerStarted","Data":"63cc71a11951ad1ea56b3a8d5ee5234ddfae9af6aee1d0873f7d0f1732aecebb"} Jul 28 19:37:07.097931 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.097854 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" event={"ID":"183d967b-aba7-4b25-843a-1de6360c3ad9","Type":"ContainerStarted","Data":"5b726c1e76ef613a86330ea66e23714a5595ea9c388bc97caca2b2b1310f72e4"} Jul 28 19:37:07.097931 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.097906 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" event={"ID":"183d967b-aba7-4b25-843a-1de6360c3ad9","Type":"ContainerStarted","Data":"502bfa1f2ebef310f955d1409a6b1807e8c8a2c8cf01dca641d6685bfab660b6"} Jul 28 19:37:07.107615 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.107578 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tj7qj" podStartSLOduration=3.730384371 podStartE2EDuration="22.107561735s" totalImagesPullingTime="18.377177364s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:07.107080071 +0000 UTC m=+22.814291965" watchObservedRunningTime="2026-07-28 19:37:07.107561735 +0000 UTC m=+22.814773628" Jul 28 19:37:07.108027 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.108000 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" podStartSLOduration=21.10799136 podStartE2EDuration="21.10799136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:36:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:50.064845292 +0000 UTC m=+5.772057186" watchObservedRunningTime="2026-07-28 19:37:07.10799136 +0000 UTC m=+22.815203249" Jul 28 19:37:07.123366 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.123322 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6kkrr" podStartSLOduration=3.810310707 podStartE2EDuration="22.123309806s" totalImagesPullingTime="18.312999099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:07.123187996 +0000 UTC m=+22.830399890" watchObservedRunningTime="2026-07-28 19:37:07.123309806 +0000 UTC m=+22.830521698" Jul 28 19:37:07.138419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.138376 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b7h24" podStartSLOduration=7.622665189 podStartE2EDuration="22.138362112s" totalImagesPullingTime="14.515696923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:07.137799448 +0000 UTC m=+22.845011349" watchObservedRunningTime="2026-07-28 19:37:07.138362112 +0000 UTC m=+22.845574005" Jul 28 19:37:07.151519 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.151449 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-s2jhz" podStartSLOduration=3.835433699 podStartE2EDuration="22.151442101s" totalImagesPullingTime="18.316008402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:07.15128248 +0000 UTC m=+22.858494373" watchObservedRunningTime="2026-07-28 19:37:07.151442101 +0000 UTC m=+22.858653994" Jul 28 19:37:07.866506 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.866483 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 28 19:37:07.913316 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:07.913249 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:07.913430 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:07.913396 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:08.101579 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:08.101531 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" event={"ID":"6caf88fe-b980-421f-bfed-83cf7c76c49a","Type":"ContainerStarted","Data":"66ee87c69cef7f44321281a427ca75d6041ff8b75ce94ee7e1985fefba306ef7"} Jul 28 19:37:08.103186 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:08.103159 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s7ftg" event={"ID":"1be64912-89f6-4b48-a5d2-9a3aac8acdcf","Type":"ContainerStarted","Data":"2b95a7a08cb0426b05edfd9012f6cbae028325d0bab557e17e94676eef36688b"} Jul 28 19:37:08.118533 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:08.118483 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-s7ftg" podStartSLOduration=4.81118225 podStartE2EDuration="23.118462113s" totalImagesPullingTime="18.307279863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:08.118310841 +0000 UTC m=+23.825522734" watchObservedRunningTime="2026-07-28 19:37:08.118462113 +0000 UTC m=+23.825674007" Jul 28 19:37:08.118813 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:08.118790 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mq2zb" podStartSLOduration=4.772756917 podStartE2EDuration="23.118783928s" totalImagesPullingTime="18.346027011s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:07.200465885 +0000 UTC m=+22.907677779" watchObservedRunningTime="2026-07-28 19:37:08.118783928 +0000 UTC m=+23.825995821" Jul 28 19:37:08.554585 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:08.554543 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s2jhz_7e437f25-b6ba-42b8-b1c1-9245e8a86bc4/dns-node-resolver/0.log" Jul 28 19:37:08.773837 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:08.773721 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T19:37:07.866499834Z","UUID":"0a5178a8-0ebc-4007-b8ab-39ea2adfcf77","Handler":null,"Name":"","Endpoint":""} Jul 28 19:37:08.775404 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:08.775380 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 28 19:37:08.775519 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:08.775426 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 28 19:37:08.913153 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:08.913131 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:08.913271 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:08.913249 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:08.913619 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:08.913601 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:08.913749 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:08.913724 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:08.926268 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:08.926242 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b7h24" Jul 28 19:37:08.926710 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:08.926691 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b7h24" Jul 28 19:37:09.107676 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:09.107623 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" event={"ID":"6caf88fe-b980-421f-bfed-83cf7c76c49a","Type":"ContainerStarted","Data":"bd1b93a084135313cf5f3ccddbb5c27e705f41151ad4a894e9e64f13fc5025d9"} Jul 28 19:37:09.200015 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:09.199885 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b7h24" Jul 28 19:37:09.200497 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:09.200466 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b7h24" Jul 28 19:37:09.217256 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:09.217213 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q5hp4" podStartSLOduration=3.673657593 podStartE2EDuration="24.217198398s" totalImagesPullingTime="20.543540805s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:36:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:09.124410931 +0000 UTC m=+24.831622826" watchObservedRunningTime="2026-07-28 19:37:09.217198398 +0000 UTC m=+24.924410295" Jul 28 19:37:09.336003 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:09.335972 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6kkrr_095953aa-1212-406f-b21e-7b649915699d/node-ca/0.log" Jul 28 19:37:09.912541 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:09.912318 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:09.912797 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:09.912688 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:10.113890 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:10.113855 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" event={"ID":"183d967b-aba7-4b25-843a-1de6360c3ad9","Type":"ContainerStarted","Data":"00519bf84efaff9446559b421783751f5f9b99e4fc3cc1b370932c482564bcc7"} Jul 28 19:37:10.912619 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:10.912587 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:10.912855 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:10.912744 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:10.912855 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:10.912800 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:10.912972 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:10.912911 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:11.912945 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:11.912783 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:11.913517 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:11.913029 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:12.121983 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:12.121951 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" event={"ID":"183d967b-aba7-4b25-843a-1de6360c3ad9","Type":"ContainerStarted","Data":"1f482811b6e2a3d46ea3538086cdd40fe7c6f0dd76c663ea6d8aacec76a4c65a"} Jul 28 19:37:12.122208 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:12.122194 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:37:12.122321 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:12.122244 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:37:12.123773 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:12.123746 2564 generic.go:356] "Generic (PLEG): container finished" podID="9e333f58-159f-4604-aefa-55d3aeb458b6" containerID="5e5e8cfbf8f445e7e2043379f079db276fe9415159020019829f1d750e3be62e" exitCode=0 Jul 28 19:37:12.123902 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:12.123801 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpbk6" event={"ID":"9e333f58-159f-4604-aefa-55d3aeb458b6","Type":"ContainerDied","Data":"5e5e8cfbf8f445e7e2043379f079db276fe9415159020019829f1d750e3be62e"} Jul 28 19:37:12.138458 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:12.138436 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:37:12.154694 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:12.154657 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" podStartSLOduration=8.789706782 podStartE2EDuration="27.154623615s" totalImagesPullingTime="18.364916833s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:12.154590599 +0000 UTC m=+27.861802491" watchObservedRunningTime="2026-07-28 19:37:12.154623615 +0000 UTC m=+27.861835502" Jul 28 19:37:12.912343 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:12.912280 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:12.912343 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:12.912309 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:12.912532 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:12.912386 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:12.912532 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:12.912500 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:13.127666 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:13.126602 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:37:13.141454 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:13.141433 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:37:13.912787 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:13.912754 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:13.913011 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:13.912875 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:14.129457 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:14.129419 2564 generic.go:356] "Generic (PLEG): container finished" podID="9e333f58-159f-4604-aefa-55d3aeb458b6" containerID="54e07a5b4f9c99b3a56dc1b6b90fb0edd124a748c84ad1fec7a6a6b3220684ef" exitCode=0 Jul 28 19:37:14.129919 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:14.129507 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpbk6" event={"ID":"9e333f58-159f-4604-aefa-55d3aeb458b6","Type":"ContainerDied","Data":"54e07a5b4f9c99b3a56dc1b6b90fb0edd124a748c84ad1fec7a6a6b3220684ef"} Jul 28 19:37:14.912975 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:14.912944 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:14.913152 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:14.913010 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:14.913152 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:14.913024 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:14.913152 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:14.913079 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:15.912787 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:15.912756 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:15.913175 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:15.912872 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:16.135804 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:16.135772 2564 generic.go:356] "Generic (PLEG): container finished" podID="9e333f58-159f-4604-aefa-55d3aeb458b6" containerID="0c88f8394a92a35ca2aa69e8db303b9b4651ea8564d6df75576c36af4f187a26" exitCode=0 Jul 28 19:37:16.135953 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:16.135831 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpbk6" event={"ID":"9e333f58-159f-4604-aefa-55d3aeb458b6","Type":"ContainerDied","Data":"0c88f8394a92a35ca2aa69e8db303b9b4651ea8564d6df75576c36af4f187a26"} Jul 28 19:37:16.913141 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:16.913110 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:16.913141 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:16.913147 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:16.913673 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:16.913232 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:16.913673 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:16.913331 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:17.913090 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:17.913057 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:17.913279 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:17.913182 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:18.456256 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:18.456223 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:18.456445 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:18.456272 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs\") pod \"network-metrics-daemon-q6qbg\" (UID: \"842143d0-262e-4779-90aa-50bb8b33633b\") " pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:18.456445 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:18.456372 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 19:37:18.456445 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:18.456422 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs podName:842143d0-262e-4779-90aa-50bb8b33633b nodeName:}" failed. No retries permitted until 2026-07-28 19:37:50.456409209 +0000 UTC m=+66.163621086 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs") pod "network-metrics-daemon-q6qbg" (UID: "842143d0-262e-4779-90aa-50bb8b33633b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 19:37:18.456584 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:18.456372 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 19:37:18.456584 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:18.456488 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret podName:b5208c81-86ee-4635-9dd3-137393805762 nodeName:}" failed. No retries permitted until 2026-07-28 19:37:50.456476272 +0000 UTC m=+66.163688156 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret") pod "global-pull-secret-syncer-2xb59" (UID: "b5208c81-86ee-4635-9dd3-137393805762") : object "kube-system"/"original-pull-secret" not registered Jul 28 19:37:18.556861 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:18.556826 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdwfh\" (UniqueName: \"kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh\") pod \"network-check-target-p4mcm\" (UID: \"7442e5a0-6ae1-43a9-b67e-ada4e63d3a41\") " pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:18.557019 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:18.556944 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 19:37:18.557019 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:18.556969 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 19:37:18.557019 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:18.556981 2564 projected.go:196] Error preparing data for projected volume kube-api-access-mdwfh for pod openshift-network-diagnostics/network-check-target-p4mcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 19:37:18.557140 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:18.557038 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh podName:7442e5a0-6ae1-43a9-b67e-ada4e63d3a41 nodeName:}" failed. No retries permitted until 2026-07-28 19:37:50.557022854 +0000 UTC m=+66.264234725 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mdwfh" (UniqueName: "kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh") pod "network-check-target-p4mcm" (UID: "7442e5a0-6ae1-43a9-b67e-ada4e63d3a41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 19:37:18.912469 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:18.912435 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:18.912670 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:18.912477 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:18.912670 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:18.912576 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:18.912805 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:18.912722 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:19.912685 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:19.912653 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:19.913126 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:19.912790 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:20.148288 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:20.148255 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpbk6" event={"ID":"9e333f58-159f-4604-aefa-55d3aeb458b6","Type":"ContainerStarted","Data":"d7b6163e16938b5a34b86fd0c1563a6ae36d72496309ec229d0b2ddf7f22195a"} Jul 28 19:37:20.912501 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:20.912467 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:20.912719 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:20.912474 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:20.912719 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:20.912564 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:20.912719 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:20.912666 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:21.152510 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:21.152476 2564 generic.go:356] "Generic (PLEG): container finished" podID="9e333f58-159f-4604-aefa-55d3aeb458b6" containerID="d7b6163e16938b5a34b86fd0c1563a6ae36d72496309ec229d0b2ddf7f22195a" exitCode=0 Jul 28 19:37:21.152688 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:21.152516 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpbk6" event={"ID":"9e333f58-159f-4604-aefa-55d3aeb458b6","Type":"ContainerDied","Data":"d7b6163e16938b5a34b86fd0c1563a6ae36d72496309ec229d0b2ddf7f22195a"} Jul 28 19:37:21.912571 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:21.912546 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:21.912753 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:21.912653 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:22.157080 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:22.157046 2564 generic.go:356] "Generic (PLEG): container finished" podID="9e333f58-159f-4604-aefa-55d3aeb458b6" containerID="a956bd93f23bcd4752dea49e0abf0fb5345e3c12da9f67a866419396d1cacc07" exitCode=0 Jul 28 19:37:22.157246 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:22.157097 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpbk6" event={"ID":"9e333f58-159f-4604-aefa-55d3aeb458b6","Type":"ContainerDied","Data":"a956bd93f23bcd4752dea49e0abf0fb5345e3c12da9f67a866419396d1cacc07"} Jul 28 19:37:22.912827 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:22.912794 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:22.913178 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:22.912836 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:22.913178 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:22.912911 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:22.913178 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:22.913024 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:23.162781 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:23.162590 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gpbk6" event={"ID":"9e333f58-159f-4604-aefa-55d3aeb458b6","Type":"ContainerStarted","Data":"4f012c29742dbde85ab05233d996240f8d201e27a5d7760ed129b290a7663b37"} Jul 28 19:37:23.186538 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:23.186456 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gpbk6" podStartSLOduration=9.213017593 podStartE2EDuration="38.186442255s" totalImagesPullingTime="28.973424662s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:36:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:23.186118715 +0000 UTC m=+38.893330610" watchObservedRunningTime="2026-07-28 19:37:23.186442255 +0000 UTC m=+38.893654148" Jul 28 19:37:23.912560 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:23.912526 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:23.912727 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:23.912651 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:24.913553 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:24.913522 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:24.913943 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:24.913595 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:24.913943 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:24.913702 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:24.913943 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:24.913600 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:25.913244 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:25.913216 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:25.913401 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:25.913313 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:26.913335 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:26.913301 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:26.913715 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:26.913410 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:26.913715 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:26.913488 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:26.913715 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:26.913593 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:27.913191 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:27.913159 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:27.913379 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:27.913278 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:28.913160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:28.913125 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:28.913330 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:28.913224 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:28.913330 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:28.913305 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:28.913432 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:28.913410 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:29.913088 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:29.913041 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:29.913398 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:29.913177 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:30.766088 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:30.766058 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-q6qbg"] Jul 28 19:37:30.766774 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:30.766170 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:30.766774 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:30.766272 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:30.768577 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:30.768533 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2xb59"] Jul 28 19:37:30.768723 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:30.768670 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:30.768805 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:30.768781 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:30.782047 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:30.781981 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p4mcm"] Jul 28 19:37:30.782157 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:30.782080 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:30.782213 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:30.782155 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:31.912929 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:31.912891 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:31.912929 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:31.912920 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:31.913398 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:31.913024 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:31.913398 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:31.913076 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:31.913398 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:31.913154 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:31.913398 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:31.913249 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:33.913284 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:33.913253 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:33.913669 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:33.913346 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:33.913669 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:33.913351 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p4mcm" podUID="7442e5a0-6ae1-43a9-b67e-ada4e63d3a41" Jul 28 19:37:33.913669 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:33.913444 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q6qbg" podUID="842143d0-262e-4779-90aa-50bb8b33633b" Jul 28 19:37:33.913669 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:33.913350 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:33.913669 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:33.913523 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xb59" podUID="b5208c81-86ee-4635-9dd3-137393805762" Jul 28 19:37:35.095087 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.095013 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeReady" Jul 28 19:37:35.095487 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.095118 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 28 19:37:35.144923 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.144889 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zhp49"] Jul 28 19:37:35.159107 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.159084 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-m9dst"] Jul 28 19:37:35.159273 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.159252 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.162096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.162075 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 28 19:37:35.162275 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.162259 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cdlcv\"" Jul 28 19:37:35.162365 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.162349 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 28 19:37:35.162434 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.162382 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 28 19:37:35.162509 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.162495 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 28 19:37:35.173932 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.173769 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m9dst"] Jul 28 19:37:35.174043 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.173965 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zhp49"] Jul 28 19:37:35.174043 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.173869 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m9dst" Jul 28 19:37:35.176874 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.176855 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 28 19:37:35.176969 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.176945 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 28 19:37:35.176969 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.176953 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-phccc\"" Jul 28 19:37:35.177090 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.177048 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 28 19:37:35.244793 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.244743 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fd2mx"] Jul 28 19:37:35.259020 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.258997 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fd2mx"] Jul 28 19:37:35.259149 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.259092 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:35.261977 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.261952 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pkrrg\"" Jul 28 19:37:35.261977 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.261972 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 28 19:37:35.262133 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.262004 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 28 19:37:35.281786 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.281760 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zhp49\" (UID: \"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1\") " pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.281904 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.281790 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd5zt\" (UniqueName: \"kubernetes.io/projected/e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1-kube-api-access-nd5zt\") pod \"insights-runtime-extractor-zhp49\" (UID: \"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1\") " pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.281904 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.281814 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1-data-volume\") pod \"insights-runtime-extractor-zhp49\" (UID: \"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1\") " pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.281904 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.281851 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1-crio-socket\") pod \"insights-runtime-extractor-zhp49\" (UID: \"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1\") " pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.281904 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.281874 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zhp49\" (UID: \"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1\") " pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.282069 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.281930 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtrdk\" (UniqueName: \"kubernetes.io/projected/accaa0ec-2c25-42e2-9065-392d7b2a0a23-kube-api-access-xtrdk\") pod \"ingress-canary-m9dst\" (UID: \"accaa0ec-2c25-42e2-9065-392d7b2a0a23\") " pod="openshift-ingress-canary/ingress-canary-m9dst" Jul 28 19:37:35.282069 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.281956 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/accaa0ec-2c25-42e2-9065-392d7b2a0a23-cert\") pod \"ingress-canary-m9dst\" (UID: \"accaa0ec-2c25-42e2-9065-392d7b2a0a23\") " pod="openshift-ingress-canary/ingress-canary-m9dst" Jul 28 19:37:35.382552 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.382441 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zhp49\" (UID: \"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1\") " pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.382552 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.382476 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd5zt\" (UniqueName: \"kubernetes.io/projected/e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1-kube-api-access-nd5zt\") pod \"insights-runtime-extractor-zhp49\" (UID: \"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1\") " pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.382552 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.382498 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wh5q\" (UniqueName: \"kubernetes.io/projected/c17c0331-989d-43c0-9298-48a364bae22b-kube-api-access-7wh5q\") pod \"dns-default-fd2mx\" (UID: \"c17c0331-989d-43c0-9298-48a364bae22b\") " pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:35.382552 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.382519 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c17c0331-989d-43c0-9298-48a364bae22b-tmp-dir\") pod \"dns-default-fd2mx\" (UID: \"c17c0331-989d-43c0-9298-48a364bae22b\") " pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:35.382552 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.382538 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1-data-volume\") pod \"insights-runtime-extractor-zhp49\" (UID: \"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1\") " pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.382864 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.382575 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c17c0331-989d-43c0-9298-48a364bae22b-metrics-tls\") pod \"dns-default-fd2mx\" (UID: \"c17c0331-989d-43c0-9298-48a364bae22b\") " pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:35.382864 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.382720 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1-crio-socket\") pod \"insights-runtime-extractor-zhp49\" (UID: \"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1\") " pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.382864 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.382746 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zhp49\" (UID: \"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1\") " pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.382864 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.382773 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtrdk\" (UniqueName: \"kubernetes.io/projected/accaa0ec-2c25-42e2-9065-392d7b2a0a23-kube-api-access-xtrdk\") pod \"ingress-canary-m9dst\" (UID: \"accaa0ec-2c25-42e2-9065-392d7b2a0a23\") " pod="openshift-ingress-canary/ingress-canary-m9dst" Jul 28 19:37:35.382864 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.382850 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c17c0331-989d-43c0-9298-48a364bae22b-config-volume\") pod \"dns-default-fd2mx\" (UID: \"c17c0331-989d-43c0-9298-48a364bae22b\") " pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:35.383092 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.382900 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/accaa0ec-2c25-42e2-9065-392d7b2a0a23-cert\") pod \"ingress-canary-m9dst\" (UID: \"accaa0ec-2c25-42e2-9065-392d7b2a0a23\") " pod="openshift-ingress-canary/ingress-canary-m9dst" Jul 28 19:37:35.383092 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.382897 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1-crio-socket\") pod \"insights-runtime-extractor-zhp49\" (UID: \"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1\") " pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.383092 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.382909 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1-data-volume\") pod \"insights-runtime-extractor-zhp49\" (UID: \"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1\") " pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.383092 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.383081 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zhp49\" (UID: \"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1\") " pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.386979 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.386959 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/accaa0ec-2c25-42e2-9065-392d7b2a0a23-cert\") pod \"ingress-canary-m9dst\" (UID: \"accaa0ec-2c25-42e2-9065-392d7b2a0a23\") " pod="openshift-ingress-canary/ingress-canary-m9dst" Jul 28 19:37:35.391699 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.391680 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd5zt\" (UniqueName: \"kubernetes.io/projected/e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1-kube-api-access-nd5zt\") pod \"insights-runtime-extractor-zhp49\" (UID: \"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1\") " pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.391824 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.391745 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtrdk\" (UniqueName: \"kubernetes.io/projected/accaa0ec-2c25-42e2-9065-392d7b2a0a23-kube-api-access-xtrdk\") pod \"ingress-canary-m9dst\" (UID: \"accaa0ec-2c25-42e2-9065-392d7b2a0a23\") " pod="openshift-ingress-canary/ingress-canary-m9dst" Jul 28 19:37:35.392848 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.392829 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zhp49\" (UID: \"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1\") " pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.468557 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.468529 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zhp49" Jul 28 19:37:35.482652 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.482607 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m9dst" Jul 28 19:37:35.484256 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.484241 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c17c0331-989d-43c0-9298-48a364bae22b-config-volume\") pod \"dns-default-fd2mx\" (UID: \"c17c0331-989d-43c0-9298-48a364bae22b\") " pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:35.484305 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.484287 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wh5q\" (UniqueName: \"kubernetes.io/projected/c17c0331-989d-43c0-9298-48a364bae22b-kube-api-access-7wh5q\") pod \"dns-default-fd2mx\" (UID: \"c17c0331-989d-43c0-9298-48a364bae22b\") " pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:35.484349 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.484309 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c17c0331-989d-43c0-9298-48a364bae22b-tmp-dir\") pod \"dns-default-fd2mx\" (UID: \"c17c0331-989d-43c0-9298-48a364bae22b\") " pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:35.484349 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.484335 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c17c0331-989d-43c0-9298-48a364bae22b-metrics-tls\") pod \"dns-default-fd2mx\" (UID: \"c17c0331-989d-43c0-9298-48a364bae22b\") " pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:35.484610 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.484591 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c17c0331-989d-43c0-9298-48a364bae22b-tmp-dir\") pod \"dns-default-fd2mx\" (UID: \"c17c0331-989d-43c0-9298-48a364bae22b\") " pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:35.484771 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.484745 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c17c0331-989d-43c0-9298-48a364bae22b-config-volume\") pod \"dns-default-fd2mx\" (UID: \"c17c0331-989d-43c0-9298-48a364bae22b\") " pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:35.486545 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.486518 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c17c0331-989d-43c0-9298-48a364bae22b-metrics-tls\") pod \"dns-default-fd2mx\" (UID: \"c17c0331-989d-43c0-9298-48a364bae22b\") " pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:35.493245 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.493224 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wh5q\" (UniqueName: \"kubernetes.io/projected/c17c0331-989d-43c0-9298-48a364bae22b-kube-api-access-7wh5q\") pod \"dns-default-fd2mx\" (UID: \"c17c0331-989d-43c0-9298-48a364bae22b\") " pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:35.575518 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.575180 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:35.647901 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.647872 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m9dst"] Jul 28 19:37:35.650510 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.650485 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zhp49"] Jul 28 19:37:35.651935 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:37:35.651906 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaccaa0ec_2c25_42e2_9065_392d7b2a0a23.slice/crio-ce0e6628005786bc76df41a67483ca8aa542856e348d309b042c2f20a04e2857 WatchSource:0}: Error finding container ce0e6628005786bc76df41a67483ca8aa542856e348d309b042c2f20a04e2857: Status 404 returned error can't find the container with id ce0e6628005786bc76df41a67483ca8aa542856e348d309b042c2f20a04e2857 Jul 28 19:37:35.653373 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:37:35.653351 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e25d20_f2d5_4d0d_8d93_5d98fc7fc0b1.slice/crio-3e6a3042f959399a9a86fdcf9f33fcfbcd13b6122cd00fa872acd61727a1dad7 WatchSource:0}: Error finding container 3e6a3042f959399a9a86fdcf9f33fcfbcd13b6122cd00fa872acd61727a1dad7: Status 404 returned error can't find the container with id 3e6a3042f959399a9a86fdcf9f33fcfbcd13b6122cd00fa872acd61727a1dad7 Jul 28 19:37:35.708236 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.708207 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fd2mx"] Jul 28 19:37:35.712277 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:37:35.712246 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc17c0331_989d_43c0_9298_48a364bae22b.slice/crio-ba49321babb681156c3a895c78a689b068ce0cd62b20238bd226f191dee72a7c WatchSource:0}: Error finding container ba49321babb681156c3a895c78a689b068ce0cd62b20238bd226f191dee72a7c: Status 404 returned error can't find the container with id ba49321babb681156c3a895c78a689b068ce0cd62b20238bd226f191dee72a7c Jul 28 19:37:35.912540 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.912466 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:35.912540 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.912514 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:35.912722 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.912571 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:35.915749 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.915719 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 19:37:35.915851 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.915825 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 19:37:35.915927 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.915910 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jfqx7\"" Jul 28 19:37:35.915973 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.915962 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 19:37:35.916029 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.916014 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 19:37:35.916081 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:35.916065 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h6gvt\"" Jul 28 19:37:36.190999 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:36.190907 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fd2mx" event={"ID":"c17c0331-989d-43c0-9298-48a364bae22b","Type":"ContainerStarted","Data":"ba49321babb681156c3a895c78a689b068ce0cd62b20238bd226f191dee72a7c"} Jul 28 19:37:36.192241 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:36.192176 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zhp49" event={"ID":"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1","Type":"ContainerStarted","Data":"3e6a3042f959399a9a86fdcf9f33fcfbcd13b6122cd00fa872acd61727a1dad7"} Jul 28 19:37:36.193461 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:36.193440 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m9dst" event={"ID":"accaa0ec-2c25-42e2-9065-392d7b2a0a23","Type":"ContainerStarted","Data":"ce0e6628005786bc76df41a67483ca8aa542856e348d309b042c2f20a04e2857"} Jul 28 19:37:37.427029 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:37.426997 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-k2r98"] Jul 28 19:37:37.448981 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:37.448949 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-k2r98"] Jul 28 19:37:37.449232 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:37.449070 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-k2r98" Jul 28 19:37:37.453855 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:37.452698 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-mmjd9\"" Jul 28 19:37:37.453855 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:37.453561 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 28 19:37:37.501434 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:37.501334 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/aaaa8139-2b99-4500-99a9-dc350205f405-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-k2r98\" (UID: \"aaaa8139-2b99-4500-99a9-dc350205f405\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-k2r98" Jul 28 19:37:37.602772 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:37.602726 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/aaaa8139-2b99-4500-99a9-dc350205f405-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-k2r98\" (UID: \"aaaa8139-2b99-4500-99a9-dc350205f405\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-k2r98" Jul 28 19:37:37.602949 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:37.602926 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 28 19:37:37.603042 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:37.603000 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aaaa8139-2b99-4500-99a9-dc350205f405-tls-certificates podName:aaaa8139-2b99-4500-99a9-dc350205f405 nodeName:}" failed. No retries permitted until 2026-07-28 19:37:38.102979075 +0000 UTC m=+53.810190963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/aaaa8139-2b99-4500-99a9-dc350205f405-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-k2r98" (UID: "aaaa8139-2b99-4500-99a9-dc350205f405") : secret "prometheus-operator-admission-webhook-tls" not found Jul 28 19:37:38.107107 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:38.106859 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/aaaa8139-2b99-4500-99a9-dc350205f405-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-k2r98\" (UID: \"aaaa8139-2b99-4500-99a9-dc350205f405\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-k2r98" Jul 28 19:37:38.111833 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:38.111807 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/aaaa8139-2b99-4500-99a9-dc350205f405-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-k2r98\" (UID: \"aaaa8139-2b99-4500-99a9-dc350205f405\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-k2r98" Jul 28 19:37:38.364961 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:38.364873 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-k2r98" Jul 28 19:37:39.299246 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.299207 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-744df96c64-8qt8t"] Jul 28 19:37:39.310861 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.310832 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.316198 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.316151 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 28 19:37:39.316330 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.316155 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 28 19:37:39.316330 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.316227 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 28 19:37:39.316330 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.316257 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 28 19:37:39.317523 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.317296 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 28 19:37:39.317523 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.317454 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 28 19:37:39.317779 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.317647 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qnkbx\"" Jul 28 19:37:39.318979 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.318951 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744df96c64-8qt8t"] Jul 28 19:37:39.319091 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.318990 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 28 19:37:39.416567 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.416534 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-oauth-serving-cert\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.416567 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.416570 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rdgm\" (UniqueName: \"kubernetes.io/projected/e416341e-1efa-4434-99e4-6099ab8239b7-kube-api-access-7rdgm\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.416830 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.416608 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-console-config\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.416830 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.416701 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e416341e-1efa-4434-99e4-6099ab8239b7-console-serving-cert\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.416830 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.416739 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e416341e-1efa-4434-99e4-6099ab8239b7-console-oauth-config\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.416830 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.416817 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-service-ca\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.518041 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.518003 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-console-config\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.518195 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.518072 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e416341e-1efa-4434-99e4-6099ab8239b7-console-serving-cert\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.518195 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.518102 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e416341e-1efa-4434-99e4-6099ab8239b7-console-oauth-config\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.518195 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.518131 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-service-ca\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.518195 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.518171 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-oauth-serving-cert\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.518195 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.518194 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rdgm\" (UniqueName: \"kubernetes.io/projected/e416341e-1efa-4434-99e4-6099ab8239b7-kube-api-access-7rdgm\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.518919 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.518890 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-console-config\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.519071 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.518891 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-service-ca\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.519304 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.519273 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-oauth-serving-cert\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.521430 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.521394 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e416341e-1efa-4434-99e4-6099ab8239b7-console-oauth-config\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.521558 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.521437 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e416341e-1efa-4434-99e4-6099ab8239b7-console-serving-cert\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.530300 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.530280 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rdgm\" (UniqueName: \"kubernetes.io/projected/e416341e-1efa-4434-99e4-6099ab8239b7-kube-api-access-7rdgm\") pod \"console-744df96c64-8qt8t\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:39.627743 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:39.627624 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:41.745005 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:41.744903 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744df96c64-8qt8t"] Jul 28 19:37:41.752947 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:37:41.752912 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode416341e_1efa_4434_99e4_6099ab8239b7.slice/crio-60b425ee1405ae29908f6a3b25652da0f8f8b3b922916b4061f957254cf9664e WatchSource:0}: Error finding container 60b425ee1405ae29908f6a3b25652da0f8f8b3b922916b4061f957254cf9664e: Status 404 returned error can't find the container with id 60b425ee1405ae29908f6a3b25652da0f8f8b3b922916b4061f957254cf9664e Jul 28 19:37:41.763266 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:41.762935 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-k2r98"] Jul 28 19:37:41.768606 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:37:41.768576 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaaa8139_2b99_4500_99a9_dc350205f405.slice/crio-23f10b73ba909a16149504b8eb27bb76b3e66967c3c592063cf06030fcd5dbda WatchSource:0}: Error finding container 23f10b73ba909a16149504b8eb27bb76b3e66967c3c592063cf06030fcd5dbda: Status 404 returned error can't find the container with id 23f10b73ba909a16149504b8eb27bb76b3e66967c3c592063cf06030fcd5dbda Jul 28 19:37:42.213659 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:42.213604 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fd2mx" event={"ID":"c17c0331-989d-43c0-9298-48a364bae22b","Type":"ContainerStarted","Data":"1182a3bf7de45fed94162d731e45cb798a1acde0f11cf81e8b4b2c3eb702982d"} Jul 28 19:37:42.213848 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:42.213685 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fd2mx" event={"ID":"c17c0331-989d-43c0-9298-48a364bae22b","Type":"ContainerStarted","Data":"e8c1809e58da5f87f42c7c4222dca413a545139ef1843b42373dd5842bb3d2d7"} Jul 28 19:37:42.213915 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:42.213843 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:42.216717 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:42.216669 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zhp49" event={"ID":"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1","Type":"ContainerStarted","Data":"3ead529198f15f8534e2fbe1375d34e91700460f2db46b08b684fe79400aa86e"} Jul 28 19:37:42.219551 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:42.219041 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m9dst" event={"ID":"accaa0ec-2c25-42e2-9065-392d7b2a0a23","Type":"ContainerStarted","Data":"dd7a8bcc34ddd0f9bd2d70dcfdc0d0fea4e86ed717693f53bb92a983c86eb15d"} Jul 28 19:37:42.221398 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:42.221337 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744df96c64-8qt8t" event={"ID":"e416341e-1efa-4434-99e4-6099ab8239b7","Type":"ContainerStarted","Data":"60b425ee1405ae29908f6a3b25652da0f8f8b3b922916b4061f957254cf9664e"} Jul 28 19:37:42.222949 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:42.222925 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-k2r98" event={"ID":"aaaa8139-2b99-4500-99a9-dc350205f405","Type":"ContainerStarted","Data":"23f10b73ba909a16149504b8eb27bb76b3e66967c3c592063cf06030fcd5dbda"} Jul 28 19:37:42.233228 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:42.232686 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fd2mx" podStartSLOduration=1.3497105999999999 podStartE2EDuration="7.23266834s" totalImagesPullingTime="5.88295774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:42.231026173 +0000 UTC m=+57.938238062" watchObservedRunningTime="2026-07-28 19:37:42.23266834 +0000 UTC m=+57.939880230" Jul 28 19:37:42.247565 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:42.247508 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-m9dst" podStartSLOduration=1.302939663 podStartE2EDuration="7.247489484s" totalImagesPullingTime="5.944549821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:42.245668643 +0000 UTC m=+57.952880539" watchObservedRunningTime="2026-07-28 19:37:42.247489484 +0000 UTC m=+57.954701370" Jul 28 19:37:44.231346 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.231299 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-k2r98" event={"ID":"aaaa8139-2b99-4500-99a9-dc350205f405","Type":"ContainerStarted","Data":"e97b65d5cde57998ad798c781571f6d644572069f52ec934d66664ec47d7d504"} Jul 28 19:37:44.231801 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.231522 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-k2r98" Jul 28 19:37:44.237079 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.237053 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-k2r98" Jul 28 19:37:44.249792 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.249742 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-k2r98" podStartSLOduration=5.794499741 podStartE2EDuration="7.249727642s" totalImagesPullingTime="1.455227901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:44.247924513 +0000 UTC m=+59.955136408" watchObservedRunningTime="2026-07-28 19:37:44.249727642 +0000 UTC m=+59.956939535" Jul 28 19:37:44.488648 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.488560 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-5f5wf"] Jul 28 19:37:44.585119 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.585076 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-5f5wf"] Jul 28 19:37:44.585283 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.585228 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" Jul 28 19:37:44.588313 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.588279 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 28 19:37:44.588441 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.588340 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 19:37:44.589610 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.589591 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 28 19:37:44.589610 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.589604 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-pdhm4\"" Jul 28 19:37:44.589825 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.589693 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 19:37:44.589879 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.589829 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 19:37:44.658410 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.658385 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd33efa7-a1e0-4e24-8d87-384198f9b658-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-5f5wf\" (UID: \"fd33efa7-a1e0-4e24-8d87-384198f9b658\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" Jul 28 19:37:44.658524 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.658464 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlm8s\" (UniqueName: \"kubernetes.io/projected/fd33efa7-a1e0-4e24-8d87-384198f9b658-kube-api-access-tlm8s\") pod \"prometheus-operator-84d69f794d-5f5wf\" (UID: \"fd33efa7-a1e0-4e24-8d87-384198f9b658\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" Jul 28 19:37:44.658575 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.658523 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fd33efa7-a1e0-4e24-8d87-384198f9b658-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-5f5wf\" (UID: \"fd33efa7-a1e0-4e24-8d87-384198f9b658\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" Jul 28 19:37:44.658575 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.658547 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd33efa7-a1e0-4e24-8d87-384198f9b658-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-5f5wf\" (UID: \"fd33efa7-a1e0-4e24-8d87-384198f9b658\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" Jul 28 19:37:44.759065 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.758987 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fd33efa7-a1e0-4e24-8d87-384198f9b658-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-5f5wf\" (UID: \"fd33efa7-a1e0-4e24-8d87-384198f9b658\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" Jul 28 19:37:44.759065 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.759028 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd33efa7-a1e0-4e24-8d87-384198f9b658-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-5f5wf\" (UID: \"fd33efa7-a1e0-4e24-8d87-384198f9b658\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" Jul 28 19:37:44.759065 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.759063 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd33efa7-a1e0-4e24-8d87-384198f9b658-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-5f5wf\" (UID: \"fd33efa7-a1e0-4e24-8d87-384198f9b658\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" Jul 28 19:37:44.759299 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.759125 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlm8s\" (UniqueName: \"kubernetes.io/projected/fd33efa7-a1e0-4e24-8d87-384198f9b658-kube-api-access-tlm8s\") pod \"prometheus-operator-84d69f794d-5f5wf\" (UID: \"fd33efa7-a1e0-4e24-8d87-384198f9b658\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" Jul 28 19:37:44.762012 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.761988 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 28 19:37:44.762135 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.762023 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 19:37:44.762135 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.761990 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 28 19:37:44.769194 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:44.769176 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 28 19:37:44.769340 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:44.769243 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd33efa7-a1e0-4e24-8d87-384198f9b658-prometheus-operator-tls podName:fd33efa7-a1e0-4e24-8d87-384198f9b658 nodeName:}" failed. No retries permitted until 2026-07-28 19:37:45.269224147 +0000 UTC m=+60.976436035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/fd33efa7-a1e0-4e24-8d87-384198f9b658-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-5f5wf" (UID: "fd33efa7-a1e0-4e24-8d87-384198f9b658") : secret "prometheus-operator-tls" not found Jul 28 19:37:44.769340 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.769303 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 19:37:44.769967 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.769936 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd33efa7-a1e0-4e24-8d87-384198f9b658-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-5f5wf\" (UID: \"fd33efa7-a1e0-4e24-8d87-384198f9b658\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" Jul 28 19:37:44.777958 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.777933 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd33efa7-a1e0-4e24-8d87-384198f9b658-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-5f5wf\" (UID: \"fd33efa7-a1e0-4e24-8d87-384198f9b658\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" Jul 28 19:37:44.779746 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.779728 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 19:37:44.790387 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:44.790322 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlm8s\" (UniqueName: \"kubernetes.io/projected/fd33efa7-a1e0-4e24-8d87-384198f9b658-kube-api-access-tlm8s\") pod \"prometheus-operator-84d69f794d-5f5wf\" (UID: \"fd33efa7-a1e0-4e24-8d87-384198f9b658\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" Jul 28 19:37:45.146983 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:45.146953 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vl4ft" Jul 28 19:37:45.364209 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:45.364172 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fd33efa7-a1e0-4e24-8d87-384198f9b658-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-5f5wf\" (UID: \"fd33efa7-a1e0-4e24-8d87-384198f9b658\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" Jul 28 19:37:45.366674 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:45.366629 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fd33efa7-a1e0-4e24-8d87-384198f9b658-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-5f5wf\" (UID: \"fd33efa7-a1e0-4e24-8d87-384198f9b658\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" Jul 28 19:37:45.498960 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:45.498877 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-pdhm4\"" Jul 28 19:37:45.506297 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:45.506274 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" Jul 28 19:37:46.795026 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:46.795003 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-5f5wf"] Jul 28 19:37:46.798044 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:37:46.798020 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd33efa7_a1e0_4e24_8d87_384198f9b658.slice/crio-177fbb51cb91b2baa3d78fec39349dd2dfba104898ecb89a878a6ff1eff8298f WatchSource:0}: Error finding container 177fbb51cb91b2baa3d78fec39349dd2dfba104898ecb89a878a6ff1eff8298f: Status 404 returned error can't find the container with id 177fbb51cb91b2baa3d78fec39349dd2dfba104898ecb89a878a6ff1eff8298f Jul 28 19:37:47.245434 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:47.245396 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zhp49" event={"ID":"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1","Type":"ContainerStarted","Data":"714967d141a80ce1b0e77ffbf40d75fd0c85c94285d0c9373006bb8d43955f40"} Jul 28 19:37:47.246784 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:47.246752 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" event={"ID":"fd33efa7-a1e0-4e24-8d87-384198f9b658","Type":"ContainerStarted","Data":"177fbb51cb91b2baa3d78fec39349dd2dfba104898ecb89a878a6ff1eff8298f"} Jul 28 19:37:47.248376 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:47.248353 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744df96c64-8qt8t" event={"ID":"e416341e-1efa-4434-99e4-6099ab8239b7","Type":"ContainerStarted","Data":"287bf55a2294e31875c6b7c6040290e141975e9f271fa3b1f80a8142404fc3c9"} Jul 28 19:37:47.266044 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:47.266005 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-744df96c64-8qt8t" podStartSLOduration=3.354146217 podStartE2EDuration="8.26599289s" totalImagesPullingTime="4.911846673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:47.265731458 +0000 UTC m=+62.972943354" watchObservedRunningTime="2026-07-28 19:37:47.26599289 +0000 UTC m=+62.973204783" Jul 28 19:37:48.255340 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:48.255298 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" event={"ID":"fd33efa7-a1e0-4e24-8d87-384198f9b658","Type":"ContainerStarted","Data":"755ef7880b6b07d1189a224586211a50539df38bdf2c213702afa4c1850cf190"} Jul 28 19:37:48.888787 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:48.888752 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8468dc4969-vcchl"] Jul 28 19:37:48.893065 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:48.893044 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:48.901618 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:48.901577 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 28 19:37:48.903453 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:48.903430 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8468dc4969-vcchl"] Jul 28 19:37:48.994011 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:48.993975 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-trusted-ca-bundle\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:48.994178 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:48.994040 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8fee5448-3e24-4c09-b99c-d90705f807f7-console-oauth-config\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:48.994178 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:48.994062 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-console-config\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:48.994178 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:48.994086 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8fee5448-3e24-4c09-b99c-d90705f807f7-console-serving-cert\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:48.994178 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:48.994109 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-oauth-serving-cert\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:48.994178 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:48.994130 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-service-ca\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:48.994470 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:48.994233 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwcxq\" (UniqueName: \"kubernetes.io/projected/8fee5448-3e24-4c09-b99c-d90705f807f7-kube-api-access-rwcxq\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.095458 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.095418 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-trusted-ca-bundle\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.095667 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.095475 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8fee5448-3e24-4c09-b99c-d90705f807f7-console-oauth-config\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.095667 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.095495 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-console-config\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.095667 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.095519 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8fee5448-3e24-4c09-b99c-d90705f807f7-console-serving-cert\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.095841 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.095768 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-oauth-serving-cert\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.095841 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.095816 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-service-ca\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.095948 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.095864 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwcxq\" (UniqueName: \"kubernetes.io/projected/8fee5448-3e24-4c09-b99c-d90705f807f7-kube-api-access-rwcxq\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.096347 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.096300 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-console-config\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.096489 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.096464 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-trusted-ca-bundle\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.096540 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.096513 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-oauth-serving-cert\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.096602 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.096571 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-service-ca\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.098172 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.098152 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8fee5448-3e24-4c09-b99c-d90705f807f7-console-serving-cert\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.098596 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.098576 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8fee5448-3e24-4c09-b99c-d90705f807f7-console-oauth-config\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.105074 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.105057 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwcxq\" (UniqueName: \"kubernetes.io/projected/8fee5448-3e24-4c09-b99c-d90705f807f7-kube-api-access-rwcxq\") pod \"console-8468dc4969-vcchl\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.204606 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.204520 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:49.261473 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.261436 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" event={"ID":"fd33efa7-a1e0-4e24-8d87-384198f9b658","Type":"ContainerStarted","Data":"b74d631b99810da1fc624f828292f67d039c2da6875082274c948f68c0987f35"} Jul 28 19:37:49.279002 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.278958 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-5f5wf" podStartSLOduration=3.929244292 podStartE2EDuration="5.278939217s" totalImagesPullingTime="1.349694925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:49.278724375 +0000 UTC m=+64.985936272" watchObservedRunningTime="2026-07-28 19:37:49.278939217 +0000 UTC m=+64.986151109" Jul 28 19:37:49.628366 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.628321 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:49.628538 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.628440 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:37:49.629939 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.629903 2564 patch_prober.go:28] interesting pod/console-744df96c64-8qt8t container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.10:8443/health\": dial tcp 10.134.0.10:8443: connect: connection refused" start-of-body= Jul 28 19:37:49.630069 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:49.629962 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-744df96c64-8qt8t" podUID="e416341e-1efa-4434-99e4-6099ab8239b7" containerName="console" probeResult="failure" output="Get \"https://10.134.0.10:8443/health\": dial tcp 10.134.0.10:8443: connect: connection refused" Jul 28 19:37:50.391467 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.391443 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8468dc4969-vcchl"] Jul 28 19:37:50.394551 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:37:50.394525 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fee5448_3e24_4c09_b99c_d90705f807f7.slice/crio-6c9e40804b05cada75c3b10721c244303f4704f20cd62c827f4c1c84c8e01dd3 WatchSource:0}: Error finding container 6c9e40804b05cada75c3b10721c244303f4704f20cd62c827f4c1c84c8e01dd3: Status 404 returned error can't find the container with id 6c9e40804b05cada75c3b10721c244303f4704f20cd62c827f4c1c84c8e01dd3 Jul 28 19:37:50.510466 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.510430 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:50.510624 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.510556 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs\") pod \"network-metrics-daemon-q6qbg\" (UID: \"842143d0-262e-4779-90aa-50bb8b33633b\") " pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:50.514565 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.514171 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 19:37:50.514565 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.514418 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 19:37:50.522908 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.522877 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b5208c81-86ee-4635-9dd3-137393805762-original-pull-secret\") pod \"global-pull-secret-syncer-2xb59\" (UID: \"b5208c81-86ee-4635-9dd3-137393805762\") " pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:50.523012 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.522954 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/842143d0-262e-4779-90aa-50bb8b33633b-metrics-certs\") pod \"network-metrics-daemon-q6qbg\" (UID: \"842143d0-262e-4779-90aa-50bb8b33633b\") " pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:50.611607 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.611575 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdwfh\" (UniqueName: \"kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh\") pod \"network-check-target-p4mcm\" (UID: \"7442e5a0-6ae1-43a9-b67e-ada4e63d3a41\") " pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:50.614346 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.614329 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 19:37:50.622777 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.622742 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xb59" Jul 28 19:37:50.625253 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.625235 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 19:37:50.632603 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.632587 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h6gvt\"" Jul 28 19:37:50.634921 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.634900 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdwfh\" (UniqueName: \"kubernetes.io/projected/7442e5a0-6ae1-43a9-b67e-ada4e63d3a41-kube-api-access-mdwfh\") pod \"network-check-target-p4mcm\" (UID: \"7442e5a0-6ae1-43a9-b67e-ada4e63d3a41\") " pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:50.638183 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.638161 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q6qbg" Jul 28 19:37:50.758110 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.758083 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2xb59"] Jul 28 19:37:50.760869 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:37:50.760837 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5208c81_86ee_4635_9dd3_137393805762.slice/crio-606ca20c4a0bb4304bb803c136622e257b0ec238110559772be1e3fbf7ff6850 WatchSource:0}: Error finding container 606ca20c4a0bb4304bb803c136622e257b0ec238110559772be1e3fbf7ff6850: Status 404 returned error can't find the container with id 606ca20c4a0bb4304bb803c136622e257b0ec238110559772be1e3fbf7ff6850 Jul 28 19:37:50.779205 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.779181 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-q6qbg"] Jul 28 19:37:50.781174 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:37:50.781152 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod842143d0_262e_4779_90aa_50bb8b33633b.slice/crio-190e17409059481358e785d0f7d6c90659c579db7afb69717355ea0ef59c1bc6 WatchSource:0}: Error finding container 190e17409059481358e785d0f7d6c90659c579db7afb69717355ea0ef59c1bc6: Status 404 returned error can't find the container with id 190e17409059481358e785d0f7d6c90659c579db7afb69717355ea0ef59c1bc6 Jul 28 19:37:50.883211 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.883127 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m"] Jul 28 19:37:50.887753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.887733 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" Jul 28 19:37:50.891227 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.891201 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 28 19:37:50.891508 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.891487 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-pqs49\"" Jul 28 19:37:50.893974 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.893954 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 28 19:37:50.900618 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.900598 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m"] Jul 28 19:37:50.916942 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.916919 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj"] Jul 28 19:37:50.919887 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.919871 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-99bqt"] Jul 28 19:37:50.920002 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.919986 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:50.922807 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.922788 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:50.924770 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.924748 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 28 19:37:50.924770 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.924761 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-65rd9\"" Jul 28 19:37:50.924918 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.924770 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 28 19:37:50.925453 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.925435 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pwz2m\"" Jul 28 19:37:50.925546 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.925442 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 28 19:37:50.925948 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.925928 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 28 19:37:50.926074 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.925956 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 28 19:37:50.926514 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.926499 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 28 19:37:50.936901 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.936882 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jfqx7\"" Jul 28 19:37:50.938852 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.938832 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj"] Jul 28 19:37:50.943406 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:50.943382 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:51.015065 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grqkj\" (UniqueName: \"kubernetes.io/projected/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-kube-api-access-grqkj\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.015216 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015069 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cab676a-1e9b-408d-af0b-71d1dcca13f1-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.015216 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015088 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-node-exporter-wtmp\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.015216 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015107 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2cab676a-1e9b-408d-af0b-71d1dcca13f1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.015216 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015196 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.015438 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015225 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ndrq\" (UniqueName: \"kubernetes.io/projected/c73bd02b-752c-4648-8c0a-2c62a30980fb-kube-api-access-9ndrq\") pod \"openshift-state-metrics-b8f49f5c6-fcn9m\" (UID: \"c73bd02b-752c-4648-8c0a-2c62a30980fb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" Jul 28 19:37:51.015438 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015262 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-node-exporter-tls\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.015438 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015288 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2cab676a-1e9b-408d-af0b-71d1dcca13f1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.015438 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015317 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-root\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.015438 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015335 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c73bd02b-752c-4648-8c0a-2c62a30980fb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-fcn9m\" (UID: \"c73bd02b-752c-4648-8c0a-2c62a30980fb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" Jul 28 19:37:51.015438 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015354 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-sys\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.015438 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015382 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c73bd02b-752c-4648-8c0a-2c62a30980fb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-fcn9m\" (UID: \"c73bd02b-752c-4648-8c0a-2c62a30980fb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" Jul 28 19:37:51.015438 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015405 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-metrics-client-ca\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.015438 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015423 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-node-exporter-textfile\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.015936 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015443 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2cab676a-1e9b-408d-af0b-71d1dcca13f1-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.015936 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015470 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-node-exporter-accelerators-collector-config\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.015936 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015493 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2cab676a-1e9b-408d-af0b-71d1dcca13f1-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.015936 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015509 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c73bd02b-752c-4648-8c0a-2c62a30980fb-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-fcn9m\" (UID: \"c73bd02b-752c-4648-8c0a-2c62a30980fb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" Jul 28 19:37:51.015936 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.015527 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99czm\" (UniqueName: \"kubernetes.io/projected/2cab676a-1e9b-408d-af0b-71d1dcca13f1-kube-api-access-99czm\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.068990 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.068969 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p4mcm"] Jul 28 19:37:51.071303 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:37:51.071273 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7442e5a0_6ae1_43a9_b67e_ada4e63d3a41.slice/crio-b4334b03fa6d59a9cfea32bba7a0e23d32d89c8801796f6074f3df0f552800cd WatchSource:0}: Error finding container b4334b03fa6d59a9cfea32bba7a0e23d32d89c8801796f6074f3df0f552800cd: Status 404 returned error can't find the container with id b4334b03fa6d59a9cfea32bba7a0e23d32d89c8801796f6074f3df0f552800cd Jul 28 19:37:51.115938 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.115904 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-node-exporter-wtmp\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.116096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.115946 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2cab676a-1e9b-408d-af0b-71d1dcca13f1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.116096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.115968 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.116096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.115986 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ndrq\" (UniqueName: \"kubernetes.io/projected/c73bd02b-752c-4648-8c0a-2c62a30980fb-kube-api-access-9ndrq\") pod \"openshift-state-metrics-b8f49f5c6-fcn9m\" (UID: \"c73bd02b-752c-4648-8c0a-2c62a30980fb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" Jul 28 19:37:51.116096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116008 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-node-exporter-tls\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.116096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116026 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2cab676a-1e9b-408d-af0b-71d1dcca13f1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.116096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116048 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-root\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.116096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116064 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c73bd02b-752c-4648-8c0a-2c62a30980fb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-fcn9m\" (UID: \"c73bd02b-752c-4648-8c0a-2c62a30980fb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" Jul 28 19:37:51.116443 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116097 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-node-exporter-wtmp\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.116443 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116163 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-root\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.116443 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116196 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-sys\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.116443 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116245 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c73bd02b-752c-4648-8c0a-2c62a30980fb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-fcn9m\" (UID: \"c73bd02b-752c-4648-8c0a-2c62a30980fb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" Jul 28 19:37:51.116443 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116278 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-metrics-client-ca\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.116443 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116309 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-node-exporter-textfile\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.116443 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116338 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2cab676a-1e9b-408d-af0b-71d1dcca13f1-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.116443 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116364 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-node-exporter-accelerators-collector-config\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.116443 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116395 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2cab676a-1e9b-408d-af0b-71d1dcca13f1-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.116443 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116421 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c73bd02b-752c-4648-8c0a-2c62a30980fb-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-fcn9m\" (UID: \"c73bd02b-752c-4648-8c0a-2c62a30980fb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" Jul 28 19:37:51.116999 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116462 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99czm\" (UniqueName: \"kubernetes.io/projected/2cab676a-1e9b-408d-af0b-71d1dcca13f1-kube-api-access-99czm\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.116999 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116487 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grqkj\" (UniqueName: \"kubernetes.io/projected/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-kube-api-access-grqkj\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.116999 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116511 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cab676a-1e9b-408d-af0b-71d1dcca13f1-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.116999 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:51.116622 2564 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 28 19:37:51.116999 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:51.116707 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cab676a-1e9b-408d-af0b-71d1dcca13f1-kube-state-metrics-tls podName:2cab676a-1e9b-408d-af0b-71d1dcca13f1 nodeName:}" failed. No retries permitted until 2026-07-28 19:37:51.61668666 +0000 UTC m=+67.323898597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2cab676a-1e9b-408d-af0b-71d1dcca13f1-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-tthgj" (UID: "2cab676a-1e9b-408d-af0b-71d1dcca13f1") : secret "kube-state-metrics-tls" not found Jul 28 19:37:51.116999 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116336 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-sys\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.116999 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.116906 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-metrics-client-ca\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.117346 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.117042 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2cab676a-1e9b-408d-af0b-71d1dcca13f1-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.117346 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.117173 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2cab676a-1e9b-408d-af0b-71d1dcca13f1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.117346 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.117263 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2cab676a-1e9b-408d-af0b-71d1dcca13f1-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.117346 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.117312 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-node-exporter-textfile\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.117543 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.117386 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-node-exporter-accelerators-collector-config\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.117995 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.117970 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c73bd02b-752c-4648-8c0a-2c62a30980fb-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-fcn9m\" (UID: \"c73bd02b-752c-4648-8c0a-2c62a30980fb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" Jul 28 19:37:51.118722 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.118692 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.118915 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.118898 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c73bd02b-752c-4648-8c0a-2c62a30980fb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-fcn9m\" (UID: \"c73bd02b-752c-4648-8c0a-2c62a30980fb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" Jul 28 19:37:51.119378 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.119353 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-node-exporter-tls\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.119473 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.119353 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2cab676a-1e9b-408d-af0b-71d1dcca13f1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.119473 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.119427 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c73bd02b-752c-4648-8c0a-2c62a30980fb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-fcn9m\" (UID: \"c73bd02b-752c-4648-8c0a-2c62a30980fb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" Jul 28 19:37:51.125600 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.125574 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99czm\" (UniqueName: \"kubernetes.io/projected/2cab676a-1e9b-408d-af0b-71d1dcca13f1-kube-api-access-99czm\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.125956 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.125938 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ndrq\" (UniqueName: \"kubernetes.io/projected/c73bd02b-752c-4648-8c0a-2c62a30980fb-kube-api-access-9ndrq\") pod \"openshift-state-metrics-b8f49f5c6-fcn9m\" (UID: \"c73bd02b-752c-4648-8c0a-2c62a30980fb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" Jul 28 19:37:51.126161 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.126145 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grqkj\" (UniqueName: \"kubernetes.io/projected/8e38cfa6-41d0-4f5c-b27f-1fb28751700a-kube-api-access-grqkj\") pod \"node-exporter-99bqt\" (UID: \"8e38cfa6-41d0-4f5c-b27f-1fb28751700a\") " pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.195995 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.195934 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" Jul 28 19:37:51.235603 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.235571 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-99bqt" Jul 28 19:37:51.248883 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:37:51.248852 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e38cfa6_41d0_4f5c_b27f_1fb28751700a.slice/crio-af2eea5ea9dd94fd1bba83fddd7dad33e4bc725a221b025a2a9f6a0c183c0a9a WatchSource:0}: Error finding container af2eea5ea9dd94fd1bba83fddd7dad33e4bc725a221b025a2a9f6a0c183c0a9a: Status 404 returned error can't find the container with id af2eea5ea9dd94fd1bba83fddd7dad33e4bc725a221b025a2a9f6a0c183c0a9a Jul 28 19:37:51.271117 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.271004 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-99bqt" event={"ID":"8e38cfa6-41d0-4f5c-b27f-1fb28751700a","Type":"ContainerStarted","Data":"af2eea5ea9dd94fd1bba83fddd7dad33e4bc725a221b025a2a9f6a0c183c0a9a"} Jul 28 19:37:51.272366 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.272303 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2xb59" event={"ID":"b5208c81-86ee-4635-9dd3-137393805762","Type":"ContainerStarted","Data":"606ca20c4a0bb4304bb803c136622e257b0ec238110559772be1e3fbf7ff6850"} Jul 28 19:37:51.273955 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.273932 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8468dc4969-vcchl" event={"ID":"8fee5448-3e24-4c09-b99c-d90705f807f7","Type":"ContainerStarted","Data":"a1ea7630d525183c4ca8073fe0b3518c0bb7db1fa6e3949054c22731cccb3680"} Jul 28 19:37:51.274067 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.273974 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8468dc4969-vcchl" event={"ID":"8fee5448-3e24-4c09-b99c-d90705f807f7","Type":"ContainerStarted","Data":"6c9e40804b05cada75c3b10721c244303f4704f20cd62c827f4c1c84c8e01dd3"} Jul 28 19:37:51.277991 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.277454 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zhp49" event={"ID":"e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1","Type":"ContainerStarted","Data":"58471507f8e8d8007e7bdd4fe81fc5f951ad161bbc0ee5522ac7139bda07f020"} Jul 28 19:37:51.279120 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.279072 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q6qbg" event={"ID":"842143d0-262e-4779-90aa-50bb8b33633b","Type":"ContainerStarted","Data":"190e17409059481358e785d0f7d6c90659c579db7afb69717355ea0ef59c1bc6"} Jul 28 19:37:51.280413 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.280392 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p4mcm" event={"ID":"7442e5a0-6ae1-43a9-b67e-ada4e63d3a41","Type":"ContainerStarted","Data":"b4334b03fa6d59a9cfea32bba7a0e23d32d89c8801796f6074f3df0f552800cd"} Jul 28 19:37:51.293187 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.293139 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8468dc4969-vcchl" podStartSLOduration=3.293121741 podStartE2EDuration="3.293121741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:51.291423499 +0000 UTC m=+66.998635417" watchObservedRunningTime="2026-07-28 19:37:51.293121741 +0000 UTC m=+67.000333635" Jul 28 19:37:51.348735 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.348669 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zhp49" podStartSLOduration=1.826764203 podStartE2EDuration="16.348644895s" totalImagesPullingTime="14.521880692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:37:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:51.312988662 +0000 UTC m=+67.020200556" watchObservedRunningTime="2026-07-28 19:37:51.348644895 +0000 UTC m=+67.055856780" Jul 28 19:37:51.349429 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.349395 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m"] Jul 28 19:37:51.354512 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:37:51.354480 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73bd02b_752c_4648_8c0a_2c62a30980fb.slice/crio-583575eac94fee9668f31410bd17721aaefaf4d01b5c523c89f0f7b0c5126f4f WatchSource:0}: Error finding container 583575eac94fee9668f31410bd17721aaefaf4d01b5c523c89f0f7b0c5126f4f: Status 404 returned error can't find the container with id 583575eac94fee9668f31410bd17721aaefaf4d01b5c523c89f0f7b0c5126f4f Jul 28 19:37:51.622387 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.622348 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cab676a-1e9b-408d-af0b-71d1dcca13f1-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.625376 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.625348 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2cab676a-1e9b-408d-af0b-71d1dcca13f1-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-tthgj\" (UID: \"2cab676a-1e9b-408d-af0b-71d1dcca13f1\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.830835 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.830802 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" Jul 28 19:37:51.952136 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.952104 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 19:37:51.971671 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.971588 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 19:37:51.971818 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.971727 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:51.974959 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.974907 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 19:37:51.975092 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.975028 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 19:37:51.975246 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.975228 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 19:37:51.975323 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.975287 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 19:37:51.975323 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.975299 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 19:37:51.975664 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.975533 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 19:37:51.975664 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.975543 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 19:37:51.975664 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.975586 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 19:37:51.975942 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.975925 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-p6wxl\"" Jul 28 19:37:51.975942 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.975932 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 19:37:51.988562 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:51.988538 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj"] Jul 28 19:37:52.025311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.025232 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.025474 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.025329 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnjbj\" (UniqueName: \"kubernetes.io/projected/d01ee066-80cc-410b-be3d-67a1bda72148-kube-api-access-bnjbj\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.025474 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.025435 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.025474 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.025470 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d01ee066-80cc-410b-be3d-67a1bda72148-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.025613 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.025490 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d01ee066-80cc-410b-be3d-67a1bda72148-config-out\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.025613 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.025589 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d01ee066-80cc-410b-be3d-67a1bda72148-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.025745 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.025617 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.025745 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.025660 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.025745 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.025685 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-web-config\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.025884 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.025761 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d01ee066-80cc-410b-be3d-67a1bda72148-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.025884 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.025835 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d01ee066-80cc-410b-be3d-67a1bda72148-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.025884 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.025866 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.026020 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.025901 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-config-volume\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.108378 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:37:52.108294 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cab676a_1e9b_408d_af0b_71d1dcca13f1.slice/crio-2da52740ea20eb0cbf61a64279d5e26bc82e1dcdc1d78bed97a21beaf7473bb9 WatchSource:0}: Error finding container 2da52740ea20eb0cbf61a64279d5e26bc82e1dcdc1d78bed97a21beaf7473bb9: Status 404 returned error can't find the container with id 2da52740ea20eb0cbf61a64279d5e26bc82e1dcdc1d78bed97a21beaf7473bb9 Jul 28 19:37:52.126489 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.126455 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d01ee066-80cc-410b-be3d-67a1bda72148-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.126608 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.126504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.126608 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.126532 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.126608 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.126557 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-web-config\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.126608 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.126589 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d01ee066-80cc-410b-be3d-67a1bda72148-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.126858 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.126619 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d01ee066-80cc-410b-be3d-67a1bda72148-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.126858 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.126667 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.126858 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.126704 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-config-volume\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.126858 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.126739 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.126858 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.126763 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnjbj\" (UniqueName: \"kubernetes.io/projected/d01ee066-80cc-410b-be3d-67a1bda72148-kube-api-access-bnjbj\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.126858 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.126798 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.126858 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.126825 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d01ee066-80cc-410b-be3d-67a1bda72148-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.126858 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.126851 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d01ee066-80cc-410b-be3d-67a1bda72148-config-out\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.128152 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:52.127476 2564 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 28 19:37:52.128152 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:52.127549 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-main-tls podName:d01ee066-80cc-410b-be3d-67a1bda72148 nodeName:}" failed. No retries permitted until 2026-07-28 19:37:52.627528317 +0000 UTC m=+68.334740203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148") : secret "alertmanager-main-tls" not found Jul 28 19:37:52.128152 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:52.127838 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d01ee066-80cc-410b-be3d-67a1bda72148-alertmanager-trusted-ca-bundle podName:d01ee066-80cc-410b-be3d-67a1bda72148 nodeName:}" failed. No retries permitted until 2026-07-28 19:37:52.627820129 +0000 UTC m=+68.335032016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/d01ee066-80cc-410b-be3d-67a1bda72148-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148") : configmap references non-existent config key: ca-bundle.crt Jul 28 19:37:52.128152 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.128051 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d01ee066-80cc-410b-be3d-67a1bda72148-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.129506 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.129340 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d01ee066-80cc-410b-be3d-67a1bda72148-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.132247 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.132204 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-config-volume\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.132339 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.132315 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.132648 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.132562 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.133323 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.133301 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d01ee066-80cc-410b-be3d-67a1bda72148-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.133892 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.133871 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d01ee066-80cc-410b-be3d-67a1bda72148-config-out\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.133951 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.133927 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.135477 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.135451 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-web-config\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.138565 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.138522 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.140597 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.140471 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnjbj\" (UniqueName: \"kubernetes.io/projected/d01ee066-80cc-410b-be3d-67a1bda72148-kube-api-access-bnjbj\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.230267 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.229557 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fd2mx" Jul 28 19:37:52.288112 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.288079 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" event={"ID":"2cab676a-1e9b-408d-af0b-71d1dcca13f1","Type":"ContainerStarted","Data":"2da52740ea20eb0cbf61a64279d5e26bc82e1dcdc1d78bed97a21beaf7473bb9"} Jul 28 19:37:52.292662 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.292053 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" event={"ID":"c73bd02b-752c-4648-8c0a-2c62a30980fb","Type":"ContainerStarted","Data":"5feb0a4d7972a3c583df8f8a63ad44b2ffab4f168d59e97843a44c7a5f824bb3"} Jul 28 19:37:52.292662 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.292113 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" event={"ID":"c73bd02b-752c-4648-8c0a-2c62a30980fb","Type":"ContainerStarted","Data":"c5ed214b7d0f624771189666db866931d8292ac87de21e09817f8c2f080ffbf6"} Jul 28 19:37:52.292662 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.292145 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" event={"ID":"c73bd02b-752c-4648-8c0a-2c62a30980fb","Type":"ContainerStarted","Data":"583575eac94fee9668f31410bd17721aaefaf4d01b5c523c89f0f7b0c5126f4f"} Jul 28 19:37:52.631487 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.631405 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d01ee066-80cc-410b-be3d-67a1bda72148-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.631487 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.631476 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:52.631987 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:52.631660 2564 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 28 19:37:52.631987 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:37:52.631727 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-main-tls podName:d01ee066-80cc-410b-be3d-67a1bda72148 nodeName:}" failed. No retries permitted until 2026-07-28 19:37:53.631706772 +0000 UTC m=+69.338918648 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148") : secret "alertmanager-main-tls" not found Jul 28 19:37:52.632885 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:52.632838 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d01ee066-80cc-410b-be3d-67a1bda72148-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:53.298767 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.298670 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q6qbg" event={"ID":"842143d0-262e-4779-90aa-50bb8b33633b","Type":"ContainerStarted","Data":"fa6fbf7d809b60055ddb7c7067b38d3523c36b22b3302f35dbb78d9a628650f4"} Jul 28 19:37:53.298767 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.298728 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q6qbg" event={"ID":"842143d0-262e-4779-90aa-50bb8b33633b","Type":"ContainerStarted","Data":"a04232b17d3bc6174400020e2d4f4f6451d64fb9abe5ce2a5ee28fa33f9e8b31"} Jul 28 19:37:53.316524 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.316095 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-q6qbg" podStartSLOduration=66.962194973 podStartE2EDuration="1m8.316074078s" totalImagesPullingTime="1.353879105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:53.314899641 +0000 UTC m=+69.022111535" watchObservedRunningTime="2026-07-28 19:37:53.316074078 +0000 UTC m=+69.023285972" Jul 28 19:37:53.643410 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.643326 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:53.646405 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.646375 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:53.785246 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.785206 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:37:53.856139 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.856108 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-68f88d7b94-cw5hp"] Jul 28 19:37:53.859838 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.859484 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:53.867513 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.867155 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 28 19:37:53.867513 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.867424 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1n280opq14vo2\"" Jul 28 19:37:53.867813 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.867716 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 28 19:37:53.868398 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.867926 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-fv7gv\"" Jul 28 19:37:53.868398 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.868220 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 28 19:37:53.868533 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.868463 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 28 19:37:53.868698 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.868680 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 28 19:37:53.881284 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.881120 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-68f88d7b94-cw5hp"] Jul 28 19:37:53.947055 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.947016 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6blxb\" (UniqueName: \"kubernetes.io/projected/91e557d8-c007-485b-96a2-a9f334099b1c-kube-api-access-6blxb\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:53.947228 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.947085 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91e557d8-c007-485b-96a2-a9f334099b1c-metrics-client-ca\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:53.947228 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.947105 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:53.947228 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.947122 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-grpc-tls\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:53.947228 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.947147 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:53.947228 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.947173 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:53.947228 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.947195 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:53.947228 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.947225 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-thanos-querier-tls\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:53.999821 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:53.999792 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 19:37:54.048697 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.048661 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6blxb\" (UniqueName: \"kubernetes.io/projected/91e557d8-c007-485b-96a2-a9f334099b1c-kube-api-access-6blxb\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.048866 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.048745 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91e557d8-c007-485b-96a2-a9f334099b1c-metrics-client-ca\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.048866 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.048768 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.048866 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.048786 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-grpc-tls\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.048866 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.048809 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.048866 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.048835 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.048866 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.048863 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.049152 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.048895 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-thanos-querier-tls\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.049742 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.049715 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91e557d8-c007-485b-96a2-a9f334099b1c-metrics-client-ca\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.051815 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.051789 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.051928 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.051904 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.052052 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.052029 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.052111 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.052029 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.052232 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.052210 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-grpc-tls\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.052302 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.052280 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/91e557d8-c007-485b-96a2-a9f334099b1c-secret-thanos-querier-tls\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.057214 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.057187 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6blxb\" (UniqueName: \"kubernetes.io/projected/91e557d8-c007-485b-96a2-a9f334099b1c-kube-api-access-6blxb\") pod \"thanos-querier-68f88d7b94-cw5hp\" (UID: \"91e557d8-c007-485b-96a2-a9f334099b1c\") " pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.183651 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.183561 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:37:54.305163 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.305130 2564 generic.go:356] "Generic (PLEG): container finished" podID="8e38cfa6-41d0-4f5c-b27f-1fb28751700a" containerID="61d03ecacec4663ff59786ba01aae23a98c99f19713ab10e469e36ae804b45ac" exitCode=0 Jul 28 19:37:54.305343 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.305215 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-99bqt" event={"ID":"8e38cfa6-41d0-4f5c-b27f-1fb28751700a","Type":"ContainerDied","Data":"61d03ecacec4663ff59786ba01aae23a98c99f19713ab10e469e36ae804b45ac"} Jul 28 19:37:54.307937 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.307912 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" event={"ID":"c73bd02b-752c-4648-8c0a-2c62a30980fb","Type":"ContainerStarted","Data":"6e32f0aeaa5d0f347285ea2bb4c59497155cbec016cd22d3184068e3e68d0369"} Jul 28 19:37:54.343226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:54.343172 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fcn9m" podStartSLOduration=2.56018452 podStartE2EDuration="4.343152236s" totalImagesPullingTime="1.782967716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:54.340767774 +0000 UTC m=+70.047979668" watchObservedRunningTime="2026-07-28 19:37:54.343152236 +0000 UTC m=+70.050364129" Jul 28 19:37:55.258226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.258189 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr"] Jul 28 19:37:55.260570 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.260550 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.263458 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.263437 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 28 19:37:55.263570 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.263480 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qhs2n\"" Jul 28 19:37:55.264598 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.264487 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 28 19:37:55.264598 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.264529 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 28 19:37:55.264598 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.264582 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5v3tujret05ef\"" Jul 28 19:37:55.264851 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.264833 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 28 19:37:55.272344 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.272322 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr"] Jul 28 19:37:55.358470 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.358437 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ba941b7e-7dd8-4162-b69a-6f7495855147-secret-metrics-server-client-certs\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.358673 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.358496 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ba941b7e-7dd8-4162-b69a-6f7495855147-secret-metrics-server-tls\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.358673 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.358519 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ba941b7e-7dd8-4162-b69a-6f7495855147-audit-log\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.358673 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.358586 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ba941b7e-7dd8-4162-b69a-6f7495855147-metrics-server-audit-profiles\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.358673 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.358612 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vt5v\" (UniqueName: \"kubernetes.io/projected/ba941b7e-7dd8-4162-b69a-6f7495855147-kube-api-access-7vt5v\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.358673 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.358658 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba941b7e-7dd8-4162-b69a-6f7495855147-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.358952 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.358710 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ba941b7e-7dd8-4162-b69a-6f7495855147-client-ca-bundle\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.459847 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.459812 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ba941b7e-7dd8-4162-b69a-6f7495855147-secret-metrics-server-tls\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.459847 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.459853 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ba941b7e-7dd8-4162-b69a-6f7495855147-audit-log\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.460096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.459908 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ba941b7e-7dd8-4162-b69a-6f7495855147-metrics-server-audit-profiles\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.460096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.459932 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vt5v\" (UniqueName: \"kubernetes.io/projected/ba941b7e-7dd8-4162-b69a-6f7495855147-kube-api-access-7vt5v\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.460096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.459961 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba941b7e-7dd8-4162-b69a-6f7495855147-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.460096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.460036 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ba941b7e-7dd8-4162-b69a-6f7495855147-client-ca-bundle\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.460349 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.460309 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ba941b7e-7dd8-4162-b69a-6f7495855147-audit-log\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.460466 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.460384 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ba941b7e-7dd8-4162-b69a-6f7495855147-secret-metrics-server-client-certs\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.460989 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.460963 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ba941b7e-7dd8-4162-b69a-6f7495855147-metrics-server-audit-profiles\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.461124 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.461099 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba941b7e-7dd8-4162-b69a-6f7495855147-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.463371 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.463350 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ba941b7e-7dd8-4162-b69a-6f7495855147-secret-metrics-server-client-certs\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.463490 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.463427 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ba941b7e-7dd8-4162-b69a-6f7495855147-secret-metrics-server-tls\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.463490 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.463478 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ba941b7e-7dd8-4162-b69a-6f7495855147-client-ca-bundle\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.468810 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.468786 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vt5v\" (UniqueName: \"kubernetes.io/projected/ba941b7e-7dd8-4162-b69a-6f7495855147-kube-api-access-7vt5v\") pod \"metrics-server-6d4fd8f47c-z8vjr\" (UID: \"ba941b7e-7dd8-4162-b69a-6f7495855147\") " pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.571940 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.571909 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:37:55.577666 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:37:55.577622 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd01ee066_80cc_410b_be3d_67a1bda72148.slice/crio-a74e4802ce0c70abd70231a3fd58f2f660383aa93a38d36f9e086b75fc19355d WatchSource:0}: Error finding container a74e4802ce0c70abd70231a3fd58f2f660383aa93a38d36f9e086b75fc19355d: Status 404 returned error can't find the container with id a74e4802ce0c70abd70231a3fd58f2f660383aa93a38d36f9e086b75fc19355d Jul 28 19:37:55.639257 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.639222 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c"] Jul 28 19:37:55.642504 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.642483 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c" Jul 28 19:37:55.646387 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.646363 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-f2qcf\"" Jul 28 19:37:55.646572 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.646550 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 28 19:37:55.656253 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.656230 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c"] Jul 28 19:37:55.763416 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.763379 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/25774138-a7c7-43ae-a269-fd184cff3099-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-xxt9c\" (UID: \"25774138-a7c7-43ae-a269-fd184cff3099\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c" Jul 28 19:37:55.763582 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.763434 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvwvt\" (UniqueName: \"kubernetes.io/projected/25774138-a7c7-43ae-a269-fd184cff3099-kube-api-access-nvwvt\") pod \"monitoring-plugin-68b7fc9574-xxt9c\" (UID: \"25774138-a7c7-43ae-a269-fd184cff3099\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c" Jul 28 19:37:55.864562 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.864463 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/25774138-a7c7-43ae-a269-fd184cff3099-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-xxt9c\" (UID: \"25774138-a7c7-43ae-a269-fd184cff3099\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c" Jul 28 19:37:55.864562 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.864527 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvwvt\" (UniqueName: \"kubernetes.io/projected/25774138-a7c7-43ae-a269-fd184cff3099-kube-api-access-nvwvt\") pod \"monitoring-plugin-68b7fc9574-xxt9c\" (UID: \"25774138-a7c7-43ae-a269-fd184cff3099\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c" Jul 28 19:37:55.867236 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.867208 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/25774138-a7c7-43ae-a269-fd184cff3099-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-xxt9c\" (UID: \"25774138-a7c7-43ae-a269-fd184cff3099\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c" Jul 28 19:37:55.874902 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.874876 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvwvt\" (UniqueName: \"kubernetes.io/projected/25774138-a7c7-43ae-a269-fd184cff3099-kube-api-access-nvwvt\") pod \"monitoring-plugin-68b7fc9574-xxt9c\" (UID: \"25774138-a7c7-43ae-a269-fd184cff3099\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c" Jul 28 19:37:55.952828 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:55.952797 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c" Jul 28 19:37:56.317157 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.317126 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d01ee066-80cc-410b-be3d-67a1bda72148","Type":"ContainerStarted","Data":"a74e4802ce0c70abd70231a3fd58f2f660383aa93a38d36f9e086b75fc19355d"} Jul 28 19:37:56.717625 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.717593 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-744df96c64-8qt8t"] Jul 28 19:37:56.760038 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.756957 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65964b56b7-brxdg"] Jul 28 19:37:56.762316 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.762252 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.778132 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.765917 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65964b56b7-brxdg"] Jul 28 19:37:56.830394 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.830100 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr"] Jul 28 19:37:56.852622 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.852590 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c"] Jul 28 19:37:56.867460 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.867324 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-68f88d7b94-cw5hp"] Jul 28 19:37:56.880054 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.879380 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-trusted-ca-bundle\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.880054 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.879453 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-oauth-config\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.880054 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.879487 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-serving-cert\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.880054 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.879512 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-oauth-serving-cert\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.880054 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.879546 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbkfw\" (UniqueName: \"kubernetes.io/projected/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-kube-api-access-gbkfw\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.880054 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.879597 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-config\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.880054 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.879622 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-service-ca\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.982817 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.982154 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-oauth-config\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.982817 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.982201 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-serving-cert\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.982817 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.982231 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-oauth-serving-cert\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.982817 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.982263 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbkfw\" (UniqueName: \"kubernetes.io/projected/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-kube-api-access-gbkfw\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.982817 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.982310 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-config\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.982817 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.982336 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-service-ca\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.982817 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.982380 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-trusted-ca-bundle\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.984721 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.983444 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-oauth-serving-cert\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.984721 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.984136 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-config\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.984721 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.984685 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-service-ca\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.985187 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.985163 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-trusted-ca-bundle\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.989542 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.989514 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-oauth-config\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.991401 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.991379 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-serving-cert\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:56.993995 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:56.993955 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbkfw\" (UniqueName: \"kubernetes.io/projected/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-kube-api-access-gbkfw\") pod \"console-65964b56b7-brxdg\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:57.134746 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.134630 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:37:57.208039 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.208002 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 19:37:57.216961 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.213944 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.217929 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.217908 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 28 19:37:57.218133 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.217913 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 28 19:37:57.218302 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.218145 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 28 19:37:57.219042 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.219020 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 28 19:37:57.219326 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.219290 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 28 19:37:57.219596 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.219577 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 28 19:37:57.219847 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.219828 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 28 19:37:57.219929 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.219889 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 28 19:37:57.220577 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.220209 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 28 19:37:57.220577 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.220222 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 28 19:37:57.220577 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.220551 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 28 19:37:57.227068 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.222333 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-eq45rqmpcm345\"" Jul 28 19:37:57.227068 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.222765 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 28 19:37:57.227068 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.223764 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-s2r9v\"" Jul 28 19:37:57.227068 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.225361 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 28 19:37:57.231864 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.231843 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 19:37:57.238046 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.238026 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 28 19:37:57.286138 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.286093 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.286315 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.286227 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.286390 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.286323 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.286449 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.286408 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.286501 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.286476 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k86sd\" (UniqueName: \"kubernetes.io/projected/4a9613c2-3835-416d-bf81-089120dcca9e-kube-api-access-k86sd\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.286668 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.286578 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.286765 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.286718 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-config\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.286819 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.286766 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.286819 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.286796 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.286923 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.286857 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.286923 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.286917 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.287034 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.286966 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.287034 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.286996 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a9613c2-3835-416d-bf81-089120dcca9e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.287034 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.287023 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.287183 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.287046 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.287183 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.287112 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a9613c2-3835-416d-bf81-089120dcca9e-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.287183 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.287146 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.287340 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.287193 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.287340 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.287219 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.287340 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.287263 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.320385 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.320357 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65964b56b7-brxdg"] Jul 28 19:37:57.325684 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:37:57.325244 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda66ca1c5_3eb8_40d8_a240_c61694a7ca6e.slice/crio-e2182508d41680b5d35d520838526fe02414994a5c01e2a71945fa6da42f209e WatchSource:0}: Error finding container e2182508d41680b5d35d520838526fe02414994a5c01e2a71945fa6da42f209e: Status 404 returned error can't find the container with id e2182508d41680b5d35d520838526fe02414994a5c01e2a71945fa6da42f209e Jul 28 19:37:57.327198 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.327101 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p4mcm" event={"ID":"7442e5a0-6ae1-43a9-b67e-ada4e63d3a41","Type":"ContainerStarted","Data":"b5600623076e11fae8b69b74ecc3ac91bdf5f6f4e9abf4d3256fb09161258b7a"} Jul 28 19:37:57.327198 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.327166 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:37:57.336483 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.336457 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-99bqt" event={"ID":"8e38cfa6-41d0-4f5c-b27f-1fb28751700a","Type":"ContainerStarted","Data":"c449240411c005448d805da8288d90d783644f6c075c5d95f2740e8e1626db40"} Jul 28 19:37:57.336582 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.336513 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-99bqt" event={"ID":"8e38cfa6-41d0-4f5c-b27f-1fb28751700a","Type":"ContainerStarted","Data":"d0311553240146d320c5a39e047155b1902fea55d198e3ea27d06216a91d6636"} Jul 28 19:37:57.343783 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.343655 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2xb59" event={"ID":"b5208c81-86ee-4635-9dd3-137393805762","Type":"ContainerStarted","Data":"e872494e8690a468ff360e1bcd849a2a33992a6577b6c35cf77fa1ab036146e8"} Jul 28 19:37:57.346561 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.346284 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-p4mcm" podStartSLOduration=66.789906252 podStartE2EDuration="1m12.34626902s" totalImagesPullingTime="5.556362768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:57.34616203 +0000 UTC m=+73.053373925" watchObservedRunningTime="2026-07-28 19:37:57.34626902 +0000 UTC m=+73.053480912" Jul 28 19:37:57.348446 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.348420 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" event={"ID":"2cab676a-1e9b-408d-af0b-71d1dcca13f1","Type":"ContainerStarted","Data":"bfb29dbb31712d959782fd9dc37a830ad6a7bd351559d186d450e242753624ca"} Jul 28 19:37:57.348545 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.348489 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" event={"ID":"2cab676a-1e9b-408d-af0b-71d1dcca13f1","Type":"ContainerStarted","Data":"39520faad69505a9cef6cfc1819cafbfd0d58fbe3f4c7d6c49d7ee86162fd639"} Jul 28 19:37:57.348545 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.348513 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" event={"ID":"2cab676a-1e9b-408d-af0b-71d1dcca13f1","Type":"ContainerStarted","Data":"1682156ca5cd286a9d632e236ac8ff12f460a8c5a577a5f43f7052f0e2335795"} Jul 28 19:37:57.352795 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.352770 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c" event={"ID":"25774138-a7c7-43ae-a269-fd184cff3099","Type":"ContainerStarted","Data":"eb5ccf5743a8b99b072c6009f367a7e73ffbb1badd9eafc076c084d8584fcdbb"} Jul 28 19:37:57.355019 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.354982 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" event={"ID":"91e557d8-c007-485b-96a2-a9f334099b1c","Type":"ContainerStarted","Data":"b9e74be7ff3dc8e94215a45e418a0be0b91f8843ccb42ad4f7b7a4b105aed733"} Jul 28 19:37:57.356696 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.356671 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" event={"ID":"ba941b7e-7dd8-4162-b69a-6f7495855147","Type":"ContainerStarted","Data":"a34a004aa3099647100b0fe56eb788025522ea1ff9c76f531c509f5f4ac976ee"} Jul 28 19:37:57.369420 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.369383 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-99bqt" podStartSLOduration=5.233016075 podStartE2EDuration="7.369370894s" totalImagesPullingTime="2.045213775s" totalInitContainerRuntime="91.141044ms" isStatefulPod=true podCreationTimestamp="2026-07-28 19:37:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:57.368922346 +0000 UTC m=+73.076134250" watchObservedRunningTime="2026-07-28 19:37:57.369370894 +0000 UTC m=+73.076582784" Jul 28 19:37:57.388009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.387955 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.387991 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a9613c2-3835-416d-bf81-089120dcca9e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388163 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388018 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388163 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388043 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388163 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388069 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a9613c2-3835-416d-bf81-089120dcca9e-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388163 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388091 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388163 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388146 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388362 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388174 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388362 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388214 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388362 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388239 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388362 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388262 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388362 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388289 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388362 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388314 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388362 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388342 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k86sd\" (UniqueName: \"kubernetes.io/projected/4a9613c2-3835-416d-bf81-089120dcca9e-kube-api-access-k86sd\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388710 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388387 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388710 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388418 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-config\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388710 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388439 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388710 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388468 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388710 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388498 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.388710 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.388520 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.389307 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.389170 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.389307 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.389268 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.389467 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.389334 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.392183 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.391764 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.393722 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.393699 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.394820 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.394771 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tthgj" podStartSLOduration=2.913824575 podStartE2EDuration="7.394753093s" totalImagesPullingTime="4.480928518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:57.393512871 +0000 UTC m=+73.100724764" watchObservedRunningTime="2026-07-28 19:37:57.394753093 +0000 UTC m=+73.101964986" Jul 28 19:37:57.396011 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.395986 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.396216 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.396193 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.397408 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.396974 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.397408 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.397083 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.397408 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.397346 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.398069 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.397994 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-config\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.398170 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.398145 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.399241 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.398795 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a9613c2-3835-416d-bf81-089120dcca9e-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.399241 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.399187 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.399241 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.399193 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.400251 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.399913 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.400251 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.400189 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a9613c2-3835-416d-bf81-089120dcca9e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.400958 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.400905 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.401355 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.401332 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.404687 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.404667 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k86sd\" (UniqueName: \"kubernetes.io/projected/4a9613c2-3835-416d-bf81-089120dcca9e-kube-api-access-k86sd\") pod \"prometheus-k8s-0\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:57.537773 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:57.537730 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:37:58.030268 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:58.029733 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2xb59" podStartSLOduration=67.163136574 podStartE2EDuration="1m13.029704453s" totalImagesPullingTime="5.866567879s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:57.42019873 +0000 UTC m=+73.127410624" watchObservedRunningTime="2026-07-28 19:37:58.029704453 +0000 UTC m=+73.736916346" Jul 28 19:37:58.030268 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:58.030168 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 19:37:58.366845 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:58.365750 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerStarted","Data":"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a"} Jul 28 19:37:58.366845 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:58.365846 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerStarted","Data":"2fe49183869f820fa16cd7ff5239c87041d94ea162c1b56d181197b06ad89410"} Jul 28 19:37:58.369177 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:58.369143 2564 generic.go:356] "Generic (PLEG): container finished" podID="d01ee066-80cc-410b-be3d-67a1bda72148" containerID="a26c3896a41108ca77f336c42ebf4be7e25c53cb5d488d0df3fe1655e732d5ef" exitCode=0 Jul 28 19:37:58.369353 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:58.369279 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d01ee066-80cc-410b-be3d-67a1bda72148","Type":"ContainerDied","Data":"a26c3896a41108ca77f336c42ebf4be7e25c53cb5d488d0df3fe1655e732d5ef"} Jul 28 19:37:58.372776 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:58.372532 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65964b56b7-brxdg" event={"ID":"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e","Type":"ContainerStarted","Data":"a6296f31edab126bf43ab1849972c5b0bf4831f0de2bf629453b638def19ade2"} Jul 28 19:37:58.372776 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:58.372587 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65964b56b7-brxdg" event={"ID":"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e","Type":"ContainerStarted","Data":"e2182508d41680b5d35d520838526fe02414994a5c01e2a71945fa6da42f209e"} Jul 28 19:37:58.456450 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:58.456381 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65964b56b7-brxdg" podStartSLOduration=2.456358215 podStartE2EDuration="2.456358215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:37:58.456101503 +0000 UTC m=+74.163313421" watchObservedRunningTime="2026-07-28 19:37:58.456358215 +0000 UTC m=+74.163570107" Jul 28 19:37:59.205438 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:59.205402 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:59.205605 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:59.205475 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:59.211063 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:59.211036 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:37:59.376756 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:59.376720 2564 generic.go:356] "Generic (PLEG): container finished" podID="4a9613c2-3835-416d-bf81-089120dcca9e" containerID="84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a" exitCode=0 Jul 28 19:37:59.377201 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:59.376809 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerDied","Data":"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a"} Jul 28 19:37:59.381790 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:37:59.381762 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:38:00.384977 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:00.384898 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" event={"ID":"ba941b7e-7dd8-4162-b69a-6f7495855147","Type":"ContainerStarted","Data":"b43a20a084f0debbf9cd3a2a8a86617a19b912ce188a9fd251465d25900339a5"} Jul 28 19:38:00.386698 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:00.386667 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c" event={"ID":"25774138-a7c7-43ae-a269-fd184cff3099","Type":"ContainerStarted","Data":"2bd1eea69d6cf10ea5e4901c98c79fb312861bd8a1a68077bfea9d03c9ea130a"} Jul 28 19:38:00.387270 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:00.387249 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c" Jul 28 19:38:00.390807 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:00.390781 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" event={"ID":"91e557d8-c007-485b-96a2-a9f334099b1c","Type":"ContainerStarted","Data":"6e37a44ba078f0dbe3def5ae487fb4a17915e7fa570d4a2686838ad6172ccb50"} Jul 28 19:38:00.390913 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:00.390866 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" event={"ID":"91e557d8-c007-485b-96a2-a9f334099b1c","Type":"ContainerStarted","Data":"189b95f5bc54811a3a9c75634c226c1a9d722ffedb4b2b11c5fb1d12e882369e"} Jul 28 19:38:00.390982 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:00.390931 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" event={"ID":"91e557d8-c007-485b-96a2-a9f334099b1c","Type":"ContainerStarted","Data":"12415a6054aa7a2e51361a26363fde49e472f95ea9cf6249b067a8d7498f14da"} Jul 28 19:38:00.394649 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:00.394616 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c" Jul 28 19:38:00.415285 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:00.411108 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" podStartSLOduration=2.681743045 podStartE2EDuration="5.411089011s" totalImagesPullingTime="2.729345966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:38:00.408772413 +0000 UTC m=+76.115984310" watchObservedRunningTime="2026-07-28 19:38:00.411089011 +0000 UTC m=+76.118300906" Jul 28 19:38:00.434997 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:00.434946 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xxt9c" podStartSLOduration=2.7408934289999998 podStartE2EDuration="5.434925656s" totalImagesPullingTime="2.694032227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:38:00.433796988 +0000 UTC m=+76.141008881" watchObservedRunningTime="2026-07-28 19:38:00.434925656 +0000 UTC m=+76.142137553" Jul 28 19:38:01.399907 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:01.399871 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" event={"ID":"91e557d8-c007-485b-96a2-a9f334099b1c","Type":"ContainerStarted","Data":"7a3c3e8937c982a172515b9909fa4e95fdc5eb3b0a5a078b233af95194c08a4b"} Jul 28 19:38:01.400247 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:01.399968 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" event={"ID":"91e557d8-c007-485b-96a2-a9f334099b1c","Type":"ContainerStarted","Data":"8cbda485a707e8e5a371d2d503dc015e2692f24b58f431faaf863759d7568521"} Jul 28 19:38:01.403586 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:01.403557 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d01ee066-80cc-410b-be3d-67a1bda72148","Type":"ContainerStarted","Data":"d634bb602ae7f9a593d7fb420348e878f26ae1eb4935c825a40b4db37f0bfb43"} Jul 28 19:38:01.403691 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:01.403656 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d01ee066-80cc-410b-be3d-67a1bda72148","Type":"ContainerStarted","Data":"8755f320e7f88c2c65dbd991a3d89cef2df9f9f8f1040dc7e762679ebb60b32b"} Jul 28 19:38:01.403761 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:01.403700 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d01ee066-80cc-410b-be3d-67a1bda72148","Type":"ContainerStarted","Data":"4a88c13ecb3e6e17b89941d1af1ab909902dbe02fd524f358e011502d6c5f48d"} Jul 28 19:38:01.403761 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:01.403743 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d01ee066-80cc-410b-be3d-67a1bda72148","Type":"ContainerStarted","Data":"157dbedb565fe14a9cbaf13b4778630d672922c697dba8fd25610404f3e5bf5c"} Jul 28 19:38:01.403861 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:01.403782 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d01ee066-80cc-410b-be3d-67a1bda72148","Type":"ContainerStarted","Data":"55a2ea296f4b2aa07b771ba380b7f2ba59594c8d010de4ac234fcd40067dcb7a"} Jul 28 19:38:02.413218 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:02.413179 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" event={"ID":"91e557d8-c007-485b-96a2-a9f334099b1c","Type":"ContainerStarted","Data":"def7cd62fc80ac81e9ff2b5ca4dce6d8f5fc8bf43dfdf3dd5ea817bbfd92353a"} Jul 28 19:38:02.413706 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:02.413375 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:38:02.417184 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:02.417121 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d01ee066-80cc-410b-be3d-67a1bda72148","Type":"ContainerStarted","Data":"c5d1fe023637dba2de2defe720b9d1fb928c8a42837b20ad821130b45e7ab1aa"} Jul 28 19:38:02.448196 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:02.448133 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" podStartSLOduration=5.351907526 podStartE2EDuration="9.448112717s" totalImagesPullingTime="4.096205191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:38:02.447362082 +0000 UTC m=+78.154573973" watchObservedRunningTime="2026-07-28 19:38:02.448112717 +0000 UTC m=+78.155324611" Jul 28 19:38:02.482894 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:02.482846 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.365100238 podStartE2EDuration="11.482826621s" totalImagesPullingTime="5.101744092s" totalInitContainerRuntime="15.982291ms" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:38:02.480378478 +0000 UTC m=+78.187590398" watchObservedRunningTime="2026-07-28 19:38:02.482826621 +0000 UTC m=+78.190038516" Jul 28 19:38:03.425080 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:03.425036 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerStarted","Data":"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6"} Jul 28 19:38:03.425476 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:03.425124 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerStarted","Data":"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c"} Jul 28 19:38:03.425476 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:03.425163 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerStarted","Data":"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030"} Jul 28 19:38:03.425476 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:03.425195 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerStarted","Data":"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb"} Jul 28 19:38:03.425476 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:03.425219 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerStarted","Data":"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898"} Jul 28 19:38:03.425476 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:03.425248 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerStarted","Data":"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b"} Jul 28 19:38:03.458175 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:03.458127 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.116278639 podStartE2EDuration="6.458110507s" totalImagesPullingTime="3.313735866s" totalInitContainerRuntime="28.096002ms" isStatefulPod=false podCreationTimestamp="2026-07-28 19:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:38:03.456134168 +0000 UTC m=+79.163346088" watchObservedRunningTime="2026-07-28 19:38:03.458110507 +0000 UTC m=+79.165322398" Jul 28 19:38:07.135805 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:07.135686 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:38:07.135805 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:07.135743 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:38:07.140488 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:07.140469 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:38:07.445354 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:07.445275 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:38:07.497746 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:07.497710 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8468dc4969-vcchl"] Jul 28 19:38:07.538517 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:07.538487 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:38:08.430547 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:08.430518 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-68f88d7b94-cw5hp" Jul 28 19:38:15.572371 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:15.572336 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:38:15.572831 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:15.572398 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:38:21.759192 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:21.759129 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-744df96c64-8qt8t" podUID="e416341e-1efa-4434-99e4-6099ab8239b7" containerName="console" containerID="cri-o://287bf55a2294e31875c6b7c6040290e141975e9f271fa3b1f80a8142404fc3c9" gracePeriod=15 Jul 28 19:38:22.503689 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.503653 2564 generic.go:356] "Generic (PLEG): container finished" podID="e416341e-1efa-4434-99e4-6099ab8239b7" containerID="287bf55a2294e31875c6b7c6040290e141975e9f271fa3b1f80a8142404fc3c9" exitCode=0 Jul 28 19:38:22.503819 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.503707 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744df96c64-8qt8t" event={"ID":"e416341e-1efa-4434-99e4-6099ab8239b7","Type":"ContainerDied","Data":"287bf55a2294e31875c6b7c6040290e141975e9f271fa3b1f80a8142404fc3c9"} Jul 28 19:38:22.503819 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.503771 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744df96c64-8qt8t" event={"ID":"e416341e-1efa-4434-99e4-6099ab8239b7","Type":"ContainerDied","Data":"60b425ee1405ae29908f6a3b25652da0f8f8b3b922916b4061f957254cf9664e"} Jul 28 19:38:22.503819 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.503791 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60b425ee1405ae29908f6a3b25652da0f8f8b3b922916b4061f957254cf9664e" Jul 28 19:38:22.520482 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.520462 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:38:22.625069 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.625039 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rdgm\" (UniqueName: \"kubernetes.io/projected/e416341e-1efa-4434-99e4-6099ab8239b7-kube-api-access-7rdgm\") pod \"e416341e-1efa-4434-99e4-6099ab8239b7\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " Jul 28 19:38:22.625241 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.625096 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e416341e-1efa-4434-99e4-6099ab8239b7-console-oauth-config\") pod \"e416341e-1efa-4434-99e4-6099ab8239b7\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " Jul 28 19:38:22.625241 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.625136 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-console-config\") pod \"e416341e-1efa-4434-99e4-6099ab8239b7\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " Jul 28 19:38:22.625241 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.625165 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-service-ca\") pod \"e416341e-1efa-4434-99e4-6099ab8239b7\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " Jul 28 19:38:22.625241 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.625201 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-oauth-serving-cert\") pod \"e416341e-1efa-4434-99e4-6099ab8239b7\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " Jul 28 19:38:22.625241 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.625242 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e416341e-1efa-4434-99e4-6099ab8239b7-console-serving-cert\") pod \"e416341e-1efa-4434-99e4-6099ab8239b7\" (UID: \"e416341e-1efa-4434-99e4-6099ab8239b7\") " Jul 28 19:38:22.625622 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.625594 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-console-config" (OuterVolumeSpecName: "console-config") pod "e416341e-1efa-4434-99e4-6099ab8239b7" (UID: "e416341e-1efa-4434-99e4-6099ab8239b7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:38:22.625724 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.625597 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-service-ca" (OuterVolumeSpecName: "service-ca") pod "e416341e-1efa-4434-99e4-6099ab8239b7" (UID: "e416341e-1efa-4434-99e4-6099ab8239b7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:38:22.625724 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.625654 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e416341e-1efa-4434-99e4-6099ab8239b7" (UID: "e416341e-1efa-4434-99e4-6099ab8239b7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:38:22.627623 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.627581 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e416341e-1efa-4434-99e4-6099ab8239b7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e416341e-1efa-4434-99e4-6099ab8239b7" (UID: "e416341e-1efa-4434-99e4-6099ab8239b7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:38:22.627623 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.627618 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e416341e-1efa-4434-99e4-6099ab8239b7-kube-api-access-7rdgm" (OuterVolumeSpecName: "kube-api-access-7rdgm") pod "e416341e-1efa-4434-99e4-6099ab8239b7" (UID: "e416341e-1efa-4434-99e4-6099ab8239b7"). InnerVolumeSpecName "kube-api-access-7rdgm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 19:38:22.627798 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.627646 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e416341e-1efa-4434-99e4-6099ab8239b7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e416341e-1efa-4434-99e4-6099ab8239b7" (UID: "e416341e-1efa-4434-99e4-6099ab8239b7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:38:22.726703 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.726589 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-oauth-serving-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:38:22.726703 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.726614 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e416341e-1efa-4434-99e4-6099ab8239b7-console-serving-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:38:22.726703 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.726623 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rdgm\" (UniqueName: \"kubernetes.io/projected/e416341e-1efa-4434-99e4-6099ab8239b7-kube-api-access-7rdgm\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:38:22.726703 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.726648 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e416341e-1efa-4434-99e4-6099ab8239b7-console-oauth-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:38:22.726703 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.726657 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-console-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:38:22.726703 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:22.726665 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e416341e-1efa-4434-99e4-6099ab8239b7-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:38:23.507526 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:23.507498 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744df96c64-8qt8t" Jul 28 19:38:23.526335 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:23.526308 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-744df96c64-8qt8t"] Jul 28 19:38:23.532822 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:23.532804 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-744df96c64-8qt8t"] Jul 28 19:38:24.917545 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:24.917512 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e416341e-1efa-4434-99e4-6099ab8239b7" path="/var/lib/kubelet/pods/e416341e-1efa-4434-99e4-6099ab8239b7/volumes" Jul 28 19:38:28.375266 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:28.375223 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-p4mcm" Jul 28 19:38:32.522614 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.522567 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8468dc4969-vcchl" podUID="8fee5448-3e24-4c09-b99c-d90705f807f7" containerName="console" containerID="cri-o://a1ea7630d525183c4ca8073fe0b3518c0bb7db1fa6e3949054c22731cccb3680" gracePeriod=15 Jul 28 19:38:32.761191 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.761169 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:38:32.818793 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.818770 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwcxq\" (UniqueName: \"kubernetes.io/projected/8fee5448-3e24-4c09-b99c-d90705f807f7-kube-api-access-rwcxq\") pod \"8fee5448-3e24-4c09-b99c-d90705f807f7\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " Jul 28 19:38:32.818929 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.818814 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-console-config\") pod \"8fee5448-3e24-4c09-b99c-d90705f807f7\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " Jul 28 19:38:32.818929 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.818841 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8fee5448-3e24-4c09-b99c-d90705f807f7-console-oauth-config\") pod \"8fee5448-3e24-4c09-b99c-d90705f807f7\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " Jul 28 19:38:32.818929 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.818866 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8fee5448-3e24-4c09-b99c-d90705f807f7-console-serving-cert\") pod \"8fee5448-3e24-4c09-b99c-d90705f807f7\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " Jul 28 19:38:32.819058 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.818947 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-oauth-serving-cert\") pod \"8fee5448-3e24-4c09-b99c-d90705f807f7\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " Jul 28 19:38:32.819058 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.818979 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-service-ca\") pod \"8fee5448-3e24-4c09-b99c-d90705f807f7\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " Jul 28 19:38:32.819058 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.819010 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-trusted-ca-bundle\") pod \"8fee5448-3e24-4c09-b99c-d90705f807f7\" (UID: \"8fee5448-3e24-4c09-b99c-d90705f807f7\") " Jul 28 19:38:32.819311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.819282 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-console-config" (OuterVolumeSpecName: "console-config") pod "8fee5448-3e24-4c09-b99c-d90705f807f7" (UID: "8fee5448-3e24-4c09-b99c-d90705f807f7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:38:32.819457 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.819427 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-service-ca" (OuterVolumeSpecName: "service-ca") pod "8fee5448-3e24-4c09-b99c-d90705f807f7" (UID: "8fee5448-3e24-4c09-b99c-d90705f807f7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:38:32.819505 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.819434 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8fee5448-3e24-4c09-b99c-d90705f807f7" (UID: "8fee5448-3e24-4c09-b99c-d90705f807f7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:38:32.819552 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.819514 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8fee5448-3e24-4c09-b99c-d90705f807f7" (UID: "8fee5448-3e24-4c09-b99c-d90705f807f7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:38:32.821068 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.821038 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fee5448-3e24-4c09-b99c-d90705f807f7-kube-api-access-rwcxq" (OuterVolumeSpecName: "kube-api-access-rwcxq") pod "8fee5448-3e24-4c09-b99c-d90705f807f7" (UID: "8fee5448-3e24-4c09-b99c-d90705f807f7"). InnerVolumeSpecName "kube-api-access-rwcxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 19:38:32.821068 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.821049 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fee5448-3e24-4c09-b99c-d90705f807f7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8fee5448-3e24-4c09-b99c-d90705f807f7" (UID: "8fee5448-3e24-4c09-b99c-d90705f807f7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:38:32.821187 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.821089 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fee5448-3e24-4c09-b99c-d90705f807f7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8fee5448-3e24-4c09-b99c-d90705f807f7" (UID: "8fee5448-3e24-4c09-b99c-d90705f807f7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:38:32.919872 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.919850 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-oauth-serving-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:38:32.919872 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.919870 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:38:32.920009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.919883 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-trusted-ca-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:38:32.920009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.919893 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwcxq\" (UniqueName: \"kubernetes.io/projected/8fee5448-3e24-4c09-b99c-d90705f807f7-kube-api-access-rwcxq\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:38:32.920009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.919902 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8fee5448-3e24-4c09-b99c-d90705f807f7-console-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:38:32.920009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.919909 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8fee5448-3e24-4c09-b99c-d90705f807f7-console-oauth-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:38:32.920009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:32.919917 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8fee5448-3e24-4c09-b99c-d90705f807f7-console-serving-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:38:33.552794 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:33.552763 2564 generic.go:356] "Generic (PLEG): container finished" podID="8fee5448-3e24-4c09-b99c-d90705f807f7" containerID="a1ea7630d525183c4ca8073fe0b3518c0bb7db1fa6e3949054c22731cccb3680" exitCode=0 Jul 28 19:38:33.553168 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:33.552805 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8468dc4969-vcchl" Jul 28 19:38:33.553168 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:33.552814 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8468dc4969-vcchl" event={"ID":"8fee5448-3e24-4c09-b99c-d90705f807f7","Type":"ContainerDied","Data":"a1ea7630d525183c4ca8073fe0b3518c0bb7db1fa6e3949054c22731cccb3680"} Jul 28 19:38:33.553168 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:33.552854 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8468dc4969-vcchl" event={"ID":"8fee5448-3e24-4c09-b99c-d90705f807f7","Type":"ContainerDied","Data":"6c9e40804b05cada75c3b10721c244303f4704f20cd62c827f4c1c84c8e01dd3"} Jul 28 19:38:33.553168 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:33.552874 2564 scope.go:117] "RemoveContainer" containerID="a1ea7630d525183c4ca8073fe0b3518c0bb7db1fa6e3949054c22731cccb3680" Jul 28 19:38:33.561205 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:33.561189 2564 scope.go:117] "RemoveContainer" containerID="a1ea7630d525183c4ca8073fe0b3518c0bb7db1fa6e3949054c22731cccb3680" Jul 28 19:38:33.561469 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:38:33.561448 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1ea7630d525183c4ca8073fe0b3518c0bb7db1fa6e3949054c22731cccb3680\": container with ID starting with a1ea7630d525183c4ca8073fe0b3518c0bb7db1fa6e3949054c22731cccb3680 not found: ID does not exist" containerID="a1ea7630d525183c4ca8073fe0b3518c0bb7db1fa6e3949054c22731cccb3680" Jul 28 19:38:33.561513 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:33.561477 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1ea7630d525183c4ca8073fe0b3518c0bb7db1fa6e3949054c22731cccb3680"} err="failed to get container status \"a1ea7630d525183c4ca8073fe0b3518c0bb7db1fa6e3949054c22731cccb3680\": rpc error: code = NotFound desc = could not find container \"a1ea7630d525183c4ca8073fe0b3518c0bb7db1fa6e3949054c22731cccb3680\": container with ID starting with a1ea7630d525183c4ca8073fe0b3518c0bb7db1fa6e3949054c22731cccb3680 not found: ID does not exist" Jul 28 19:38:33.570024 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:33.570005 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8468dc4969-vcchl"] Jul 28 19:38:33.573652 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:33.573611 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8468dc4969-vcchl"] Jul 28 19:38:34.918692 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:34.918655 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fee5448-3e24-4c09-b99c-d90705f807f7" path="/var/lib/kubelet/pods/8fee5448-3e24-4c09-b99c-d90705f807f7/volumes" Jul 28 19:38:35.578042 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:35.578013 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:38:35.581748 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:35.581728 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6d4fd8f47c-z8vjr" Jul 28 19:38:57.538869 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:57.538834 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:38:57.557895 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:57.557851 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:38:57.665466 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:38:57.665443 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:11.201275 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.201242 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 19:39:11.201805 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.201525 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="alertmanager" containerID="cri-o://55a2ea296f4b2aa07b771ba380b7f2ba59594c8d010de4ac234fcd40067dcb7a" gracePeriod=120 Jul 28 19:39:11.201805 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.201619 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="kube-rbac-proxy" containerID="cri-o://8755f320e7f88c2c65dbd991a3d89cef2df9f9f8f1040dc7e762679ebb60b32b" gracePeriod=120 Jul 28 19:39:11.201805 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.201611 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="kube-rbac-proxy-web" containerID="cri-o://4a88c13ecb3e6e17b89941d1af1ab909902dbe02fd524f358e011502d6c5f48d" gracePeriod=120 Jul 28 19:39:11.201805 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.201688 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="prom-label-proxy" containerID="cri-o://c5d1fe023637dba2de2defe720b9d1fb928c8a42837b20ad821130b45e7ab1aa" gracePeriod=120 Jul 28 19:39:11.201805 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.201603 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="kube-rbac-proxy-metric" containerID="cri-o://d634bb602ae7f9a593d7fb420348e878f26ae1eb4935c825a40b4db37f0bfb43" gracePeriod=120 Jul 28 19:39:11.201805 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.201630 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="config-reloader" containerID="cri-o://157dbedb565fe14a9cbaf13b4778630d672922c697dba8fd25610404f3e5bf5c" gracePeriod=120 Jul 28 19:39:11.707850 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.707821 2564 generic.go:356] "Generic (PLEG): container finished" podID="d01ee066-80cc-410b-be3d-67a1bda72148" containerID="c5d1fe023637dba2de2defe720b9d1fb928c8a42837b20ad821130b45e7ab1aa" exitCode=0 Jul 28 19:39:11.707850 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.707844 2564 generic.go:356] "Generic (PLEG): container finished" podID="d01ee066-80cc-410b-be3d-67a1bda72148" containerID="d634bb602ae7f9a593d7fb420348e878f26ae1eb4935c825a40b4db37f0bfb43" exitCode=0 Jul 28 19:39:11.707850 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.707850 2564 generic.go:356] "Generic (PLEG): container finished" podID="d01ee066-80cc-410b-be3d-67a1bda72148" containerID="8755f320e7f88c2c65dbd991a3d89cef2df9f9f8f1040dc7e762679ebb60b32b" exitCode=0 Jul 28 19:39:11.707850 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.707856 2564 generic.go:356] "Generic (PLEG): container finished" podID="d01ee066-80cc-410b-be3d-67a1bda72148" containerID="157dbedb565fe14a9cbaf13b4778630d672922c697dba8fd25610404f3e5bf5c" exitCode=0 Jul 28 19:39:11.708096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.707861 2564 generic.go:356] "Generic (PLEG): container finished" podID="d01ee066-80cc-410b-be3d-67a1bda72148" containerID="55a2ea296f4b2aa07b771ba380b7f2ba59594c8d010de4ac234fcd40067dcb7a" exitCode=0 Jul 28 19:39:11.708096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.707889 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d01ee066-80cc-410b-be3d-67a1bda72148","Type":"ContainerDied","Data":"c5d1fe023637dba2de2defe720b9d1fb928c8a42837b20ad821130b45e7ab1aa"} Jul 28 19:39:11.708096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.707980 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d01ee066-80cc-410b-be3d-67a1bda72148","Type":"ContainerDied","Data":"d634bb602ae7f9a593d7fb420348e878f26ae1eb4935c825a40b4db37f0bfb43"} Jul 28 19:39:11.708096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.708010 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d01ee066-80cc-410b-be3d-67a1bda72148","Type":"ContainerDied","Data":"8755f320e7f88c2c65dbd991a3d89cef2df9f9f8f1040dc7e762679ebb60b32b"} Jul 28 19:39:11.708096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.708036 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d01ee066-80cc-410b-be3d-67a1bda72148","Type":"ContainerDied","Data":"157dbedb565fe14a9cbaf13b4778630d672922c697dba8fd25610404f3e5bf5c"} Jul 28 19:39:11.708096 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:11.708060 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d01ee066-80cc-410b-be3d-67a1bda72148","Type":"ContainerDied","Data":"55a2ea296f4b2aa07b771ba380b7f2ba59594c8d010de4ac234fcd40067dcb7a"} Jul 28 19:39:12.448466 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.448442 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.541645 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.541603 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy\") pod \"d01ee066-80cc-410b-be3d-67a1bda72148\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " Jul 28 19:39:12.541798 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.541665 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d01ee066-80cc-410b-be3d-67a1bda72148-tls-assets\") pod \"d01ee066-80cc-410b-be3d-67a1bda72148\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " Jul 28 19:39:12.541798 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.541774 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d01ee066-80cc-410b-be3d-67a1bda72148\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " Jul 28 19:39:12.541913 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.541804 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-config-volume\") pod \"d01ee066-80cc-410b-be3d-67a1bda72148\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " Jul 28 19:39:12.541913 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.541847 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d01ee066-80cc-410b-be3d-67a1bda72148-config-out\") pod \"d01ee066-80cc-410b-be3d-67a1bda72148\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " Jul 28 19:39:12.541913 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.541896 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy-web\") pod \"d01ee066-80cc-410b-be3d-67a1bda72148\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " Jul 28 19:39:12.542102 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.541941 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-main-tls\") pod \"d01ee066-80cc-410b-be3d-67a1bda72148\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " Jul 28 19:39:12.542102 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.541968 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-web-config\") pod \"d01ee066-80cc-410b-be3d-67a1bda72148\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " Jul 28 19:39:12.542102 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.541999 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d01ee066-80cc-410b-be3d-67a1bda72148-alertmanager-main-db\") pod \"d01ee066-80cc-410b-be3d-67a1bda72148\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " Jul 28 19:39:12.542102 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.542045 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnjbj\" (UniqueName: \"kubernetes.io/projected/d01ee066-80cc-410b-be3d-67a1bda72148-kube-api-access-bnjbj\") pod \"d01ee066-80cc-410b-be3d-67a1bda72148\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " Jul 28 19:39:12.542102 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.542069 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d01ee066-80cc-410b-be3d-67a1bda72148-metrics-client-ca\") pod \"d01ee066-80cc-410b-be3d-67a1bda72148\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " Jul 28 19:39:12.542102 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.542096 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-cluster-tls-config\") pod \"d01ee066-80cc-410b-be3d-67a1bda72148\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " Jul 28 19:39:12.542446 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.542150 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d01ee066-80cc-410b-be3d-67a1bda72148-alertmanager-trusted-ca-bundle\") pod \"d01ee066-80cc-410b-be3d-67a1bda72148\" (UID: \"d01ee066-80cc-410b-be3d-67a1bda72148\") " Jul 28 19:39:12.542507 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.542443 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d01ee066-80cc-410b-be3d-67a1bda72148-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d01ee066-80cc-410b-be3d-67a1bda72148" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 19:39:12.542507 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.542458 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d01ee066-80cc-410b-be3d-67a1bda72148-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d01ee066-80cc-410b-be3d-67a1bda72148" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:39:12.542811 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.542767 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d01ee066-80cc-410b-be3d-67a1bda72148-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d01ee066-80cc-410b-be3d-67a1bda72148" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:39:12.544996 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.544955 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d01ee066-80cc-410b-be3d-67a1bda72148-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d01ee066-80cc-410b-be3d-67a1bda72148" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 19:39:12.544996 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.544955 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d01ee066-80cc-410b-be3d-67a1bda72148" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:12.545912 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.545837 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d01ee066-80cc-410b-be3d-67a1bda72148" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:12.546158 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.546133 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-config-volume" (OuterVolumeSpecName: "config-volume") pod "d01ee066-80cc-410b-be3d-67a1bda72148" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:12.546328 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.546296 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d01ee066-80cc-410b-be3d-67a1bda72148-config-out" (OuterVolumeSpecName: "config-out") pod "d01ee066-80cc-410b-be3d-67a1bda72148" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 19:39:12.546328 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.546296 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d01ee066-80cc-410b-be3d-67a1bda72148-kube-api-access-bnjbj" (OuterVolumeSpecName: "kube-api-access-bnjbj") pod "d01ee066-80cc-410b-be3d-67a1bda72148" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148"). InnerVolumeSpecName "kube-api-access-bnjbj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 19:39:12.546449 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.546359 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "d01ee066-80cc-410b-be3d-67a1bda72148" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:12.546858 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.546829 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d01ee066-80cc-410b-be3d-67a1bda72148" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:12.551085 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.551046 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "d01ee066-80cc-410b-be3d-67a1bda72148" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:12.557880 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.557842 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-web-config" (OuterVolumeSpecName: "web-config") pod "d01ee066-80cc-410b-be3d-67a1bda72148" (UID: "d01ee066-80cc-410b-be3d-67a1bda72148"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:12.643031 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.643006 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:12.643031 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.643028 2564 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:12.643031 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.643037 2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d01ee066-80cc-410b-be3d-67a1bda72148-config-out\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:12.643226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.643046 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:12.643226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.643056 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-main-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:12.643226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.643078 2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-web-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:12.643226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.643086 2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d01ee066-80cc-410b-be3d-67a1bda72148-alertmanager-main-db\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:12.643226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.643094 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnjbj\" (UniqueName: \"kubernetes.io/projected/d01ee066-80cc-410b-be3d-67a1bda72148-kube-api-access-bnjbj\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:12.643226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.643102 2564 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d01ee066-80cc-410b-be3d-67a1bda72148-metrics-client-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:12.643226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.643109 2564 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-cluster-tls-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:12.643226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.643117 2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d01ee066-80cc-410b-be3d-67a1bda72148-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:12.643226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.643125 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d01ee066-80cc-410b-be3d-67a1bda72148-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:12.643226 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.643133 2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d01ee066-80cc-410b-be3d-67a1bda72148-tls-assets\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:12.714947 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.714912 2564 generic.go:356] "Generic (PLEG): container finished" podID="d01ee066-80cc-410b-be3d-67a1bda72148" containerID="4a88c13ecb3e6e17b89941d1af1ab909902dbe02fd524f358e011502d6c5f48d" exitCode=0 Jul 28 19:39:12.715114 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.714985 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.715114 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.714999 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d01ee066-80cc-410b-be3d-67a1bda72148","Type":"ContainerDied","Data":"4a88c13ecb3e6e17b89941d1af1ab909902dbe02fd524f358e011502d6c5f48d"} Jul 28 19:39:12.715114 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.715073 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d01ee066-80cc-410b-be3d-67a1bda72148","Type":"ContainerDied","Data":"a74e4802ce0c70abd70231a3fd58f2f660383aa93a38d36f9e086b75fc19355d"} Jul 28 19:39:12.715114 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.715108 2564 scope.go:117] "RemoveContainer" containerID="c5d1fe023637dba2de2defe720b9d1fb928c8a42837b20ad821130b45e7ab1aa" Jul 28 19:39:12.723793 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.723773 2564 scope.go:117] "RemoveContainer" containerID="d634bb602ae7f9a593d7fb420348e878f26ae1eb4935c825a40b4db37f0bfb43" Jul 28 19:39:12.730988 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.730968 2564 scope.go:117] "RemoveContainer" containerID="8755f320e7f88c2c65dbd991a3d89cef2df9f9f8f1040dc7e762679ebb60b32b" Jul 28 19:39:12.740475 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.740040 2564 scope.go:117] "RemoveContainer" containerID="4a88c13ecb3e6e17b89941d1af1ab909902dbe02fd524f358e011502d6c5f48d" Jul 28 19:39:12.741562 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.741542 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 19:39:12.745198 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.745176 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 19:39:12.748220 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.748196 2564 scope.go:117] "RemoveContainer" containerID="157dbedb565fe14a9cbaf13b4778630d672922c697dba8fd25610404f3e5bf5c" Jul 28 19:39:12.754823 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.754809 2564 scope.go:117] "RemoveContainer" containerID="55a2ea296f4b2aa07b771ba380b7f2ba59594c8d010de4ac234fcd40067dcb7a" Jul 28 19:39:12.761568 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.761554 2564 scope.go:117] "RemoveContainer" containerID="a26c3896a41108ca77f336c42ebf4be7e25c53cb5d488d0df3fe1655e732d5ef" Jul 28 19:39:12.768473 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.768453 2564 scope.go:117] "RemoveContainer" containerID="c5d1fe023637dba2de2defe720b9d1fb928c8a42837b20ad821130b45e7ab1aa" Jul 28 19:39:12.769283 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:39:12.769264 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5d1fe023637dba2de2defe720b9d1fb928c8a42837b20ad821130b45e7ab1aa\": container with ID starting with c5d1fe023637dba2de2defe720b9d1fb928c8a42837b20ad821130b45e7ab1aa not found: ID does not exist" containerID="c5d1fe023637dba2de2defe720b9d1fb928c8a42837b20ad821130b45e7ab1aa" Jul 28 19:39:12.769361 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.769291 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5d1fe023637dba2de2defe720b9d1fb928c8a42837b20ad821130b45e7ab1aa"} err="failed to get container status \"c5d1fe023637dba2de2defe720b9d1fb928c8a42837b20ad821130b45e7ab1aa\": rpc error: code = NotFound desc = could not find container \"c5d1fe023637dba2de2defe720b9d1fb928c8a42837b20ad821130b45e7ab1aa\": container with ID starting with c5d1fe023637dba2de2defe720b9d1fb928c8a42837b20ad821130b45e7ab1aa not found: ID does not exist" Jul 28 19:39:12.769361 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.769309 2564 scope.go:117] "RemoveContainer" containerID="d634bb602ae7f9a593d7fb420348e878f26ae1eb4935c825a40b4db37f0bfb43" Jul 28 19:39:12.769594 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:39:12.769576 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d634bb602ae7f9a593d7fb420348e878f26ae1eb4935c825a40b4db37f0bfb43\": container with ID starting with d634bb602ae7f9a593d7fb420348e878f26ae1eb4935c825a40b4db37f0bfb43 not found: ID does not exist" containerID="d634bb602ae7f9a593d7fb420348e878f26ae1eb4935c825a40b4db37f0bfb43" Jul 28 19:39:12.769665 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.769600 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d634bb602ae7f9a593d7fb420348e878f26ae1eb4935c825a40b4db37f0bfb43"} err="failed to get container status \"d634bb602ae7f9a593d7fb420348e878f26ae1eb4935c825a40b4db37f0bfb43\": rpc error: code = NotFound desc = could not find container \"d634bb602ae7f9a593d7fb420348e878f26ae1eb4935c825a40b4db37f0bfb43\": container with ID starting with d634bb602ae7f9a593d7fb420348e878f26ae1eb4935c825a40b4db37f0bfb43 not found: ID does not exist" Jul 28 19:39:12.769665 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.769617 2564 scope.go:117] "RemoveContainer" containerID="8755f320e7f88c2c65dbd991a3d89cef2df9f9f8f1040dc7e762679ebb60b32b" Jul 28 19:39:12.769934 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:39:12.769916 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8755f320e7f88c2c65dbd991a3d89cef2df9f9f8f1040dc7e762679ebb60b32b\": container with ID starting with 8755f320e7f88c2c65dbd991a3d89cef2df9f9f8f1040dc7e762679ebb60b32b not found: ID does not exist" containerID="8755f320e7f88c2c65dbd991a3d89cef2df9f9f8f1040dc7e762679ebb60b32b" Jul 28 19:39:12.769974 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.769940 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8755f320e7f88c2c65dbd991a3d89cef2df9f9f8f1040dc7e762679ebb60b32b"} err="failed to get container status \"8755f320e7f88c2c65dbd991a3d89cef2df9f9f8f1040dc7e762679ebb60b32b\": rpc error: code = NotFound desc = could not find container \"8755f320e7f88c2c65dbd991a3d89cef2df9f9f8f1040dc7e762679ebb60b32b\": container with ID starting with 8755f320e7f88c2c65dbd991a3d89cef2df9f9f8f1040dc7e762679ebb60b32b not found: ID does not exist" Jul 28 19:39:12.769974 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.769955 2564 scope.go:117] "RemoveContainer" containerID="4a88c13ecb3e6e17b89941d1af1ab909902dbe02fd524f358e011502d6c5f48d" Jul 28 19:39:12.770178 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:39:12.770162 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a88c13ecb3e6e17b89941d1af1ab909902dbe02fd524f358e011502d6c5f48d\": container with ID starting with 4a88c13ecb3e6e17b89941d1af1ab909902dbe02fd524f358e011502d6c5f48d not found: ID does not exist" containerID="4a88c13ecb3e6e17b89941d1af1ab909902dbe02fd524f358e011502d6c5f48d" Jul 28 19:39:12.770239 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770183 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a88c13ecb3e6e17b89941d1af1ab909902dbe02fd524f358e011502d6c5f48d"} err="failed to get container status \"4a88c13ecb3e6e17b89941d1af1ab909902dbe02fd524f358e011502d6c5f48d\": rpc error: code = NotFound desc = could not find container \"4a88c13ecb3e6e17b89941d1af1ab909902dbe02fd524f358e011502d6c5f48d\": container with ID starting with 4a88c13ecb3e6e17b89941d1af1ab909902dbe02fd524f358e011502d6c5f48d not found: ID does not exist" Jul 28 19:39:12.770239 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770201 2564 scope.go:117] "RemoveContainer" containerID="157dbedb565fe14a9cbaf13b4778630d672922c697dba8fd25610404f3e5bf5c" Jul 28 19:39:12.770239 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770190 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 19:39:12.770448 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:39:12.770428 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"157dbedb565fe14a9cbaf13b4778630d672922c697dba8fd25610404f3e5bf5c\": container with ID starting with 157dbedb565fe14a9cbaf13b4778630d672922c697dba8fd25610404f3e5bf5c not found: ID does not exist" containerID="157dbedb565fe14a9cbaf13b4778630d672922c697dba8fd25610404f3e5bf5c" Jul 28 19:39:12.770528 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770456 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"157dbedb565fe14a9cbaf13b4778630d672922c697dba8fd25610404f3e5bf5c"} err="failed to get container status \"157dbedb565fe14a9cbaf13b4778630d672922c697dba8fd25610404f3e5bf5c\": rpc error: code = NotFound desc = could not find container \"157dbedb565fe14a9cbaf13b4778630d672922c697dba8fd25610404f3e5bf5c\": container with ID starting with 157dbedb565fe14a9cbaf13b4778630d672922c697dba8fd25610404f3e5bf5c not found: ID does not exist" Jul 28 19:39:12.770528 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770479 2564 scope.go:117] "RemoveContainer" containerID="55a2ea296f4b2aa07b771ba380b7f2ba59594c8d010de4ac234fcd40067dcb7a" Jul 28 19:39:12.770618 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770546 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="prom-label-proxy" Jul 28 19:39:12.770618 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770556 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="prom-label-proxy" Jul 28 19:39:12.770618 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770567 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="kube-rbac-proxy-web" Jul 28 19:39:12.770618 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770572 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="kube-rbac-proxy-web" Jul 28 19:39:12.770618 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770587 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="kube-rbac-proxy" Jul 28 19:39:12.770618 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770593 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="kube-rbac-proxy" Jul 28 19:39:12.770618 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770601 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e416341e-1efa-4434-99e4-6099ab8239b7" containerName="console" Jul 28 19:39:12.770618 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770608 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e416341e-1efa-4434-99e4-6099ab8239b7" containerName="console" Jul 28 19:39:12.770618 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770615 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="kube-rbac-proxy-metric" Jul 28 19:39:12.770618 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770620 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="kube-rbac-proxy-metric" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770628 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="init-config-reloader" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770648 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="init-config-reloader" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770660 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fee5448-3e24-4c09-b99c-d90705f807f7" containerName="console" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770665 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fee5448-3e24-4c09-b99c-d90705f807f7" containerName="console" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770676 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="config-reloader" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770681 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="config-reloader" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770688 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="alertmanager" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770692 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="alertmanager" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:39:12.770735 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55a2ea296f4b2aa07b771ba380b7f2ba59594c8d010de4ac234fcd40067dcb7a\": container with ID starting with 55a2ea296f4b2aa07b771ba380b7f2ba59594c8d010de4ac234fcd40067dcb7a not found: ID does not exist" containerID="55a2ea296f4b2aa07b771ba380b7f2ba59594c8d010de4ac234fcd40067dcb7a" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770759 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="kube-rbac-proxy-metric" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770760 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55a2ea296f4b2aa07b771ba380b7f2ba59594c8d010de4ac234fcd40067dcb7a"} err="failed to get container status \"55a2ea296f4b2aa07b771ba380b7f2ba59594c8d010de4ac234fcd40067dcb7a\": rpc error: code = NotFound desc = could not find container \"55a2ea296f4b2aa07b771ba380b7f2ba59594c8d010de4ac234fcd40067dcb7a\": container with ID starting with 55a2ea296f4b2aa07b771ba380b7f2ba59594c8d010de4ac234fcd40067dcb7a not found: ID does not exist" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770770 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e416341e-1efa-4434-99e4-6099ab8239b7" containerName="console" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770775 2564 scope.go:117] "RemoveContainer" containerID="a26c3896a41108ca77f336c42ebf4be7e25c53cb5d488d0df3fe1655e732d5ef" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770782 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="kube-rbac-proxy-web" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770811 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="alertmanager" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770820 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="prom-label-proxy" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770829 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fee5448-3e24-4c09-b99c-d90705f807f7" containerName="console" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770837 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="kube-rbac-proxy" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.770845 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" containerName="config-reloader" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:39:12.771013 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a26c3896a41108ca77f336c42ebf4be7e25c53cb5d488d0df3fe1655e732d5ef\": container with ID starting with a26c3896a41108ca77f336c42ebf4be7e25c53cb5d488d0df3fe1655e732d5ef not found: ID does not exist" containerID="a26c3896a41108ca77f336c42ebf4be7e25c53cb5d488d0df3fe1655e732d5ef" Jul 28 19:39:12.771089 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.771029 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a26c3896a41108ca77f336c42ebf4be7e25c53cb5d488d0df3fe1655e732d5ef"} err="failed to get container status \"a26c3896a41108ca77f336c42ebf4be7e25c53cb5d488d0df3fe1655e732d5ef\": rpc error: code = NotFound desc = could not find container \"a26c3896a41108ca77f336c42ebf4be7e25c53cb5d488d0df3fe1655e732d5ef\": container with ID starting with a26c3896a41108ca77f336c42ebf4be7e25c53cb5d488d0df3fe1655e732d5ef not found: ID does not exist" Jul 28 19:39:12.774424 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.774409 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.777177 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.777157 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 19:39:12.777288 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.777180 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 19:39:12.777288 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.777181 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 19:39:12.777434 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.777420 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 19:39:12.777713 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.777697 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-p6wxl\"" Jul 28 19:39:12.777951 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.777928 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 19:39:12.777951 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.777944 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 19:39:12.778106 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.777944 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 19:39:12.778106 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.777928 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 19:39:12.782416 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.782399 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 19:39:12.787978 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.787960 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 19:39:12.844915 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.844836 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/90214329-74d3-4151-bf44-dbbd73a33767-tls-assets\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.844915 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.844873 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.844915 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.844889 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/90214329-74d3-4151-bf44-dbbd73a33767-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.844915 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.844913 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/90214329-74d3-4151-bf44-dbbd73a33767-config-out\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.845129 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.844945 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90214329-74d3-4151-bf44-dbbd73a33767-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.845129 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.844970 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.845129 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.844986 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd5lg\" (UniqueName: \"kubernetes.io/projected/90214329-74d3-4151-bf44-dbbd73a33767-kube-api-access-zd5lg\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.845129 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.845039 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-config-volume\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.845129 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.845063 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.845129 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.845087 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-web-config\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.845129 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.845117 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.845337 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.845161 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.845337 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.845192 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90214329-74d3-4151-bf44-dbbd73a33767-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.917623 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.917592 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d01ee066-80cc-410b-be3d-67a1bda72148" path="/var/lib/kubelet/pods/d01ee066-80cc-410b-be3d-67a1bda72148/volumes" Jul 28 19:39:12.946062 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.946039 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-web-config\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.946175 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.946073 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.946175 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.946099 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.946175 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.946124 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90214329-74d3-4151-bf44-dbbd73a33767-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.946175 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.946146 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/90214329-74d3-4151-bf44-dbbd73a33767-tls-assets\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.946175 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.946165 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.946380 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.946180 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/90214329-74d3-4151-bf44-dbbd73a33767-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.946380 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.946195 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/90214329-74d3-4151-bf44-dbbd73a33767-config-out\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.946380 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.946209 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90214329-74d3-4151-bf44-dbbd73a33767-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.946380 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.946238 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.947322 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.947300 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90214329-74d3-4151-bf44-dbbd73a33767-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.947471 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.947344 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90214329-74d3-4151-bf44-dbbd73a33767-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.947902 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.947797 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/90214329-74d3-4151-bf44-dbbd73a33767-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.948052 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.948034 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd5lg\" (UniqueName: \"kubernetes.io/projected/90214329-74d3-4151-bf44-dbbd73a33767-kube-api-access-zd5lg\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.948177 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.948162 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-config-volume\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.948280 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.948267 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.949207 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.949179 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.949288 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.949210 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/90214329-74d3-4151-bf44-dbbd73a33767-config-out\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.949349 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.949306 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/90214329-74d3-4151-bf44-dbbd73a33767-tls-assets\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.949349 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.949315 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.949349 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.949322 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-web-config\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.949546 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.949485 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.949703 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.949683 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.950880 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.950861 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.951108 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.951093 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/90214329-74d3-4151-bf44-dbbd73a33767-config-volume\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:12.958287 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:12.958271 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd5lg\" (UniqueName: \"kubernetes.io/projected/90214329-74d3-4151-bf44-dbbd73a33767-kube-api-access-zd5lg\") pod \"alertmanager-main-0\" (UID: \"90214329-74d3-4151-bf44-dbbd73a33767\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:13.084236 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:13.084205 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 19:39:13.213889 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:13.213864 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 19:39:13.215776 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:39:13.215749 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90214329_74d3_4151_bf44_dbbd73a33767.slice/crio-ea3bf6ee3f28824b4ff88d4998f43260f779b8c8f0e99e602857ac3a9df51643 WatchSource:0}: Error finding container ea3bf6ee3f28824b4ff88d4998f43260f779b8c8f0e99e602857ac3a9df51643: Status 404 returned error can't find the container with id ea3bf6ee3f28824b4ff88d4998f43260f779b8c8f0e99e602857ac3a9df51643 Jul 28 19:39:13.726160 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:13.726126 2564 generic.go:356] "Generic (PLEG): container finished" podID="90214329-74d3-4151-bf44-dbbd73a33767" containerID="a6f5f1a27113da34944aa17262ce5e1594055e042153b83ef829df9e6d8d76c1" exitCode=0 Jul 28 19:39:13.726664 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:13.726212 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"90214329-74d3-4151-bf44-dbbd73a33767","Type":"ContainerDied","Data":"a6f5f1a27113da34944aa17262ce5e1594055e042153b83ef829df9e6d8d76c1"} Jul 28 19:39:13.726664 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:13.726332 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"90214329-74d3-4151-bf44-dbbd73a33767","Type":"ContainerStarted","Data":"ea3bf6ee3f28824b4ff88d4998f43260f779b8c8f0e99e602857ac3a9df51643"} Jul 28 19:39:14.733081 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:14.733048 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"90214329-74d3-4151-bf44-dbbd73a33767","Type":"ContainerStarted","Data":"3a11a9146e6b609ef11d892498a2f460fcf884924503b4694c6f02c5f4b5e6e1"} Jul 28 19:39:14.733549 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:14.733123 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"90214329-74d3-4151-bf44-dbbd73a33767","Type":"ContainerStarted","Data":"4ff71ff00e57e38f9cf3fc98d69e72fb4ff6fe895301bf6e9449902aae630993"} Jul 28 19:39:14.733549 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:14.733160 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"90214329-74d3-4151-bf44-dbbd73a33767","Type":"ContainerStarted","Data":"36184b829566dd3ee9ab21259b2dd86be23d5811fd01f318c9cf86a430dc9092"} Jul 28 19:39:14.733549 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:14.733202 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"90214329-74d3-4151-bf44-dbbd73a33767","Type":"ContainerStarted","Data":"f6a7288dd0fbd2097b3d9bb96142e499fa5be4b405efab02cb31d31780e6fa22"} Jul 28 19:39:14.733549 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:14.733242 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"90214329-74d3-4151-bf44-dbbd73a33767","Type":"ContainerStarted","Data":"d7566b618fbe1137345a4189143178fbece807c07285123e68b998e084715c13"} Jul 28 19:39:14.733549 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:14.733276 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"90214329-74d3-4151-bf44-dbbd73a33767","Type":"ContainerStarted","Data":"cb8a67bfc58b3417c00278084cee9709315f6cdf1e3a2540d6ba42d84da8de22"} Jul 28 19:39:14.778793 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:14.778704 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.756922961 podStartE2EDuration="2.778683089s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.760128ms" isStatefulPod=false podCreationTimestamp="2026-07-28 19:39:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:39:14.775646226 +0000 UTC m=+150.482858112" watchObservedRunningTime="2026-07-28 19:39:14.778683089 +0000 UTC m=+150.485894982" Jul 28 19:39:15.502282 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.502249 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 19:39:15.502732 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.502538 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="prometheus" containerID="cri-o://112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b" gracePeriod=600 Jul 28 19:39:15.502732 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.502555 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="kube-rbac-proxy-thanos" containerID="cri-o://1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6" gracePeriod=600 Jul 28 19:39:15.502732 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.502582 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="config-reloader" containerID="cri-o://4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898" gracePeriod=600 Jul 28 19:39:15.502732 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.502592 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="thanos-sidecar" containerID="cri-o://5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb" gracePeriod=600 Jul 28 19:39:15.502732 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.502591 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="kube-rbac-proxy-web" containerID="cri-o://ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030" gracePeriod=600 Jul 28 19:39:15.502732 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.502609 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="kube-rbac-proxy" containerID="cri-o://c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c" gracePeriod=600 Jul 28 19:39:15.735442 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.735419 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:15.747006 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.746974 2564 generic.go:356] "Generic (PLEG): container finished" podID="4a9613c2-3835-416d-bf81-089120dcca9e" containerID="1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6" exitCode=0 Jul 28 19:39:15.747006 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.747003 2564 generic.go:356] "Generic (PLEG): container finished" podID="4a9613c2-3835-416d-bf81-089120dcca9e" containerID="c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c" exitCode=0 Jul 28 19:39:15.747206 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.747012 2564 generic.go:356] "Generic (PLEG): container finished" podID="4a9613c2-3835-416d-bf81-089120dcca9e" containerID="ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030" exitCode=0 Jul 28 19:39:15.747206 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.747021 2564 generic.go:356] "Generic (PLEG): container finished" podID="4a9613c2-3835-416d-bf81-089120dcca9e" containerID="5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb" exitCode=0 Jul 28 19:39:15.747206 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.747036 2564 generic.go:356] "Generic (PLEG): container finished" podID="4a9613c2-3835-416d-bf81-089120dcca9e" containerID="4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898" exitCode=0 Jul 28 19:39:15.747206 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.747044 2564 generic.go:356] "Generic (PLEG): container finished" podID="4a9613c2-3835-416d-bf81-089120dcca9e" containerID="112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b" exitCode=0 Jul 28 19:39:15.748205 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.747956 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerDied","Data":"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6"} Jul 28 19:39:15.748205 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.748042 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerDied","Data":"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c"} Jul 28 19:39:15.748205 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.748079 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerDied","Data":"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030"} Jul 28 19:39:15.748205 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.748096 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:15.748205 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.748115 2564 scope.go:117] "RemoveContainer" containerID="1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6" Jul 28 19:39:15.749261 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.748587 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerDied","Data":"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb"} Jul 28 19:39:15.749363 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.749297 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerDied","Data":"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898"} Jul 28 19:39:15.749363 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.749338 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerDied","Data":"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b"} Jul 28 19:39:15.749474 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.749374 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a9613c2-3835-416d-bf81-089120dcca9e","Type":"ContainerDied","Data":"2fe49183869f820fa16cd7ff5239c87041d94ea162c1b56d181197b06ad89410"} Jul 28 19:39:15.757301 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.757276 2564 scope.go:117] "RemoveContainer" containerID="c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c" Jul 28 19:39:15.767826 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.767794 2564 scope.go:117] "RemoveContainer" containerID="ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030" Jul 28 19:39:15.776651 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.776603 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-kubelet-serving-ca-bundle\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.776787 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.776675 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-trusted-ca-bundle\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.776787 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.776711 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.776787 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.776749 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-web-config\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777026 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.776807 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-tls\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777026 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.776840 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-metrics-client-ca\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777026 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.776882 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-1\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777026 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.776912 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-grpc-tls\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777026 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.776941 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k86sd\" (UniqueName: \"kubernetes.io/projected/4a9613c2-3835-416d-bf81-089120dcca9e-kube-api-access-k86sd\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777026 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.776964 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-0\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777026 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.776998 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777465 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.777036 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-thanos-prometheus-http-client-file\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777465 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.777063 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-2\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777465 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.777102 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-kube-rbac-proxy\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777465 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.777129 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-metrics-client-certs\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777465 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.777181 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-db\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777465 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.777213 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a9613c2-3835-416d-bf81-089120dcca9e-tls-assets\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777465 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.777243 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-serving-certs-ca-bundle\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777465 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.777267 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-config\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777465 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.777300 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a9613c2-3835-416d-bf81-089120dcca9e-config-out\") pod \"4a9613c2-3835-416d-bf81-089120dcca9e\" (UID: \"4a9613c2-3835-416d-bf81-089120dcca9e\") " Jul 28 19:39:15.777465 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.777402 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:39:15.778396 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.778378 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.778938 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.778905 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:39:15.779671 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.779618 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:39:15.780127 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.780101 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:39:15.780399 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.780374 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 19:39:15.780929 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.780885 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:39:15.782119 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.781785 2564 scope.go:117] "RemoveContainer" containerID="5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb" Jul 28 19:39:15.782555 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.782289 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:39:15.783140 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.782018 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:39:15.784051 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.784016 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:15.788106 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.788054 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:15.788323 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.788284 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9613c2-3835-416d-bf81-089120dcca9e-config-out" (OuterVolumeSpecName: "config-out") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 19:39:15.789822 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.789777 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:15.790057 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.790031 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:15.790130 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.790053 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:15.790130 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.790110 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a9613c2-3835-416d-bf81-089120dcca9e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 19:39:15.790254 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.790152 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a9613c2-3835-416d-bf81-089120dcca9e-kube-api-access-k86sd" (OuterVolumeSpecName: "kube-api-access-k86sd") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "kube-api-access-k86sd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 19:39:15.790320 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.790253 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:15.790320 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.790301 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:15.792337 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.792313 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-config" (OuterVolumeSpecName: "config") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:15.793307 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.793291 2564 scope.go:117] "RemoveContainer" containerID="4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898" Jul 28 19:39:15.800963 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.800939 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-web-config" (OuterVolumeSpecName: "web-config") pod "4a9613c2-3835-416d-bf81-089120dcca9e" (UID: "4a9613c2-3835-416d-bf81-089120dcca9e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:15.802769 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.802748 2564 scope.go:117] "RemoveContainer" containerID="112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b" Jul 28 19:39:15.811328 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.811278 2564 scope.go:117] "RemoveContainer" containerID="84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a" Jul 28 19:39:15.820324 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.820299 2564 scope.go:117] "RemoveContainer" containerID="1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6" Jul 28 19:39:15.820616 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:39:15.820589 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6\": container with ID starting with 1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6 not found: ID does not exist" containerID="1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6" Jul 28 19:39:15.820715 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.820627 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6"} err="failed to get container status \"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6\": rpc error: code = NotFound desc = could not find container \"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6\": container with ID starting with 1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6 not found: ID does not exist" Jul 28 19:39:15.820715 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.820670 2564 scope.go:117] "RemoveContainer" containerID="c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c" Jul 28 19:39:15.820973 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:39:15.820951 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c\": container with ID starting with c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c not found: ID does not exist" containerID="c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c" Jul 28 19:39:15.821071 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.820980 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c"} err="failed to get container status \"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c\": rpc error: code = NotFound desc = could not find container \"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c\": container with ID starting with c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c not found: ID does not exist" Jul 28 19:39:15.821071 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.820996 2564 scope.go:117] "RemoveContainer" containerID="ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030" Jul 28 19:39:15.821253 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:39:15.821229 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030\": container with ID starting with ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030 not found: ID does not exist" containerID="ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030" Jul 28 19:39:15.821295 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.821261 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030"} err="failed to get container status \"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030\": rpc error: code = NotFound desc = could not find container \"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030\": container with ID starting with ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030 not found: ID does not exist" Jul 28 19:39:15.821295 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.821282 2564 scope.go:117] "RemoveContainer" containerID="5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb" Jul 28 19:39:15.821549 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:39:15.821522 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb\": container with ID starting with 5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb not found: ID does not exist" containerID="5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb" Jul 28 19:39:15.821588 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.821554 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb"} err="failed to get container status \"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb\": rpc error: code = NotFound desc = could not find container \"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb\": container with ID starting with 5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb not found: ID does not exist" Jul 28 19:39:15.821588 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.821572 2564 scope.go:117] "RemoveContainer" containerID="4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898" Jul 28 19:39:15.821882 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:39:15.821865 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898\": container with ID starting with 4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898 not found: ID does not exist" containerID="4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898" Jul 28 19:39:15.821937 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.821885 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898"} err="failed to get container status \"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898\": rpc error: code = NotFound desc = could not find container \"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898\": container with ID starting with 4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898 not found: ID does not exist" Jul 28 19:39:15.821937 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.821899 2564 scope.go:117] "RemoveContainer" containerID="112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b" Jul 28 19:39:15.822163 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:39:15.822134 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b\": container with ID starting with 112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b not found: ID does not exist" containerID="112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b" Jul 28 19:39:15.822210 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.822169 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b"} err="failed to get container status \"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b\": rpc error: code = NotFound desc = could not find container \"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b\": container with ID starting with 112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b not found: ID does not exist" Jul 28 19:39:15.822210 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.822187 2564 scope.go:117] "RemoveContainer" containerID="84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a" Jul 28 19:39:15.822447 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:39:15.822423 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a\": container with ID starting with 84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a not found: ID does not exist" containerID="84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a" Jul 28 19:39:15.822536 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.822451 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a"} err="failed to get container status \"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a\": rpc error: code = NotFound desc = could not find container \"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a\": container with ID starting with 84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a not found: ID does not exist" Jul 28 19:39:15.822536 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.822471 2564 scope.go:117] "RemoveContainer" containerID="1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6" Jul 28 19:39:15.824766 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.824248 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6"} err="failed to get container status \"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6\": rpc error: code = NotFound desc = could not find container \"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6\": container with ID starting with 1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6 not found: ID does not exist" Jul 28 19:39:15.824766 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.824273 2564 scope.go:117] "RemoveContainer" containerID="c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c" Jul 28 19:39:15.824766 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.824492 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c"} err="failed to get container status \"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c\": rpc error: code = NotFound desc = could not find container \"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c\": container with ID starting with c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c not found: ID does not exist" Jul 28 19:39:15.824766 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.824512 2564 scope.go:117] "RemoveContainer" containerID="ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030" Jul 28 19:39:15.824988 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.824771 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030"} err="failed to get container status \"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030\": rpc error: code = NotFound desc = could not find container \"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030\": container with ID starting with ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030 not found: ID does not exist" Jul 28 19:39:15.824988 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.824791 2564 scope.go:117] "RemoveContainer" containerID="5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb" Jul 28 19:39:15.825082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.825031 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb"} err="failed to get container status \"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb\": rpc error: code = NotFound desc = could not find container \"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb\": container with ID starting with 5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb not found: ID does not exist" Jul 28 19:39:15.825082 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.825053 2564 scope.go:117] "RemoveContainer" containerID="4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898" Jul 28 19:39:15.825276 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.825257 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898"} err="failed to get container status \"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898\": rpc error: code = NotFound desc = could not find container \"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898\": container with ID starting with 4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898 not found: ID does not exist" Jul 28 19:39:15.825276 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.825277 2564 scope.go:117] "RemoveContainer" containerID="112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b" Jul 28 19:39:15.825536 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.825515 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b"} err="failed to get container status \"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b\": rpc error: code = NotFound desc = could not find container \"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b\": container with ID starting with 112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b not found: ID does not exist" Jul 28 19:39:15.825623 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.825537 2564 scope.go:117] "RemoveContainer" containerID="84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a" Jul 28 19:39:15.826061 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.826032 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a"} err="failed to get container status \"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a\": rpc error: code = NotFound desc = could not find container \"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a\": container with ID starting with 84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a not found: ID does not exist" Jul 28 19:39:15.826158 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.826060 2564 scope.go:117] "RemoveContainer" containerID="1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6" Jul 28 19:39:15.826346 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.826315 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6"} err="failed to get container status \"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6\": rpc error: code = NotFound desc = could not find container \"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6\": container with ID starting with 1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6 not found: ID does not exist" Jul 28 19:39:15.826346 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.826344 2564 scope.go:117] "RemoveContainer" containerID="c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c" Jul 28 19:39:15.826618 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.826593 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c"} err="failed to get container status \"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c\": rpc error: code = NotFound desc = could not find container \"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c\": container with ID starting with c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c not found: ID does not exist" Jul 28 19:39:15.826709 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.826620 2564 scope.go:117] "RemoveContainer" containerID="ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030" Jul 28 19:39:15.826938 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.826903 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030"} err="failed to get container status \"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030\": rpc error: code = NotFound desc = could not find container \"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030\": container with ID starting with ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030 not found: ID does not exist" Jul 28 19:39:15.826982 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.826940 2564 scope.go:117] "RemoveContainer" containerID="5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb" Jul 28 19:39:15.827177 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.827158 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb"} err="failed to get container status \"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb\": rpc error: code = NotFound desc = could not find container \"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb\": container with ID starting with 5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb not found: ID does not exist" Jul 28 19:39:15.827215 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.827180 2564 scope.go:117] "RemoveContainer" containerID="4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898" Jul 28 19:39:15.827415 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.827397 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898"} err="failed to get container status \"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898\": rpc error: code = NotFound desc = could not find container \"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898\": container with ID starting with 4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898 not found: ID does not exist" Jul 28 19:39:15.827498 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.827418 2564 scope.go:117] "RemoveContainer" containerID="112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b" Jul 28 19:39:15.827687 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.827668 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b"} err="failed to get container status \"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b\": rpc error: code = NotFound desc = could not find container \"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b\": container with ID starting with 112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b not found: ID does not exist" Jul 28 19:39:15.827757 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.827687 2564 scope.go:117] "RemoveContainer" containerID="84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a" Jul 28 19:39:15.827900 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.827884 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a"} err="failed to get container status \"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a\": rpc error: code = NotFound desc = could not find container \"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a\": container with ID starting with 84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a not found: ID does not exist" Jul 28 19:39:15.827983 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.827902 2564 scope.go:117] "RemoveContainer" containerID="1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6" Jul 28 19:39:15.828130 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.828111 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6"} err="failed to get container status \"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6\": rpc error: code = NotFound desc = could not find container \"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6\": container with ID starting with 1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6 not found: ID does not exist" Jul 28 19:39:15.828179 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.828131 2564 scope.go:117] "RemoveContainer" containerID="c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c" Jul 28 19:39:15.828384 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.828362 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c"} err="failed to get container status \"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c\": rpc error: code = NotFound desc = could not find container \"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c\": container with ID starting with c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c not found: ID does not exist" Jul 28 19:39:15.828431 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.828387 2564 scope.go:117] "RemoveContainer" containerID="ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030" Jul 28 19:39:15.828581 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.828562 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030"} err="failed to get container status \"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030\": rpc error: code = NotFound desc = could not find container \"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030\": container with ID starting with ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030 not found: ID does not exist" Jul 28 19:39:15.828648 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.828584 2564 scope.go:117] "RemoveContainer" containerID="5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb" Jul 28 19:39:15.828864 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.828846 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb"} err="failed to get container status \"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb\": rpc error: code = NotFound desc = could not find container \"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb\": container with ID starting with 5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb not found: ID does not exist" Jul 28 19:39:15.828941 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.828866 2564 scope.go:117] "RemoveContainer" containerID="4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898" Jul 28 19:39:15.829126 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.829109 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898"} err="failed to get container status \"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898\": rpc error: code = NotFound desc = could not find container \"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898\": container with ID starting with 4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898 not found: ID does not exist" Jul 28 19:39:15.829199 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.829128 2564 scope.go:117] "RemoveContainer" containerID="112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b" Jul 28 19:39:15.829331 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.829312 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b"} err="failed to get container status \"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b\": rpc error: code = NotFound desc = could not find container \"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b\": container with ID starting with 112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b not found: ID does not exist" Jul 28 19:39:15.829380 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.829334 2564 scope.go:117] "RemoveContainer" containerID="84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a" Jul 28 19:39:15.829585 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.829562 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a"} err="failed to get container status \"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a\": rpc error: code = NotFound desc = could not find container \"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a\": container with ID starting with 84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a not found: ID does not exist" Jul 28 19:39:15.829585 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.829583 2564 scope.go:117] "RemoveContainer" containerID="1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6" Jul 28 19:39:15.829833 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.829815 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6"} err="failed to get container status \"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6\": rpc error: code = NotFound desc = could not find container \"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6\": container with ID starting with 1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6 not found: ID does not exist" Jul 28 19:39:15.829881 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.829835 2564 scope.go:117] "RemoveContainer" containerID="c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c" Jul 28 19:39:15.830075 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.830057 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c"} err="failed to get container status \"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c\": rpc error: code = NotFound desc = could not find container \"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c\": container with ID starting with c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c not found: ID does not exist" Jul 28 19:39:15.830147 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.830088 2564 scope.go:117] "RemoveContainer" containerID="ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030" Jul 28 19:39:15.830311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.830291 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030"} err="failed to get container status \"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030\": rpc error: code = NotFound desc = could not find container \"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030\": container with ID starting with ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030 not found: ID does not exist" Jul 28 19:39:15.830362 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.830311 2564 scope.go:117] "RemoveContainer" containerID="5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb" Jul 28 19:39:15.830551 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.830535 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb"} err="failed to get container status \"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb\": rpc error: code = NotFound desc = could not find container \"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb\": container with ID starting with 5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb not found: ID does not exist" Jul 28 19:39:15.830587 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.830551 2564 scope.go:117] "RemoveContainer" containerID="4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898" Jul 28 19:39:15.830814 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.830781 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898"} err="failed to get container status \"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898\": rpc error: code = NotFound desc = could not find container \"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898\": container with ID starting with 4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898 not found: ID does not exist" Jul 28 19:39:15.830905 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.830814 2564 scope.go:117] "RemoveContainer" containerID="112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b" Jul 28 19:39:15.831110 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.831086 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b"} err="failed to get container status \"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b\": rpc error: code = NotFound desc = could not find container \"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b\": container with ID starting with 112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b not found: ID does not exist" Jul 28 19:39:15.831168 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.831111 2564 scope.go:117] "RemoveContainer" containerID="84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a" Jul 28 19:39:15.831345 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.831326 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a"} err="failed to get container status \"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a\": rpc error: code = NotFound desc = could not find container \"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a\": container with ID starting with 84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a not found: ID does not exist" Jul 28 19:39:15.831345 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.831344 2564 scope.go:117] "RemoveContainer" containerID="1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6" Jul 28 19:39:15.831571 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.831548 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6"} err="failed to get container status \"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6\": rpc error: code = NotFound desc = could not find container \"1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6\": container with ID starting with 1db24b18e01d61ce36c466ff3ee67336a4ce10f740dc5ddb45564178ab6e28a6 not found: ID does not exist" Jul 28 19:39:15.831648 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.831572 2564 scope.go:117] "RemoveContainer" containerID="c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c" Jul 28 19:39:15.831809 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.831794 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c"} err="failed to get container status \"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c\": rpc error: code = NotFound desc = could not find container \"c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c\": container with ID starting with c317229ebfc390c12e3cc497ed5b98cb623a8a9fe9de1edb3ddf9fcd3efeb85c not found: ID does not exist" Jul 28 19:39:15.831873 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.831811 2564 scope.go:117] "RemoveContainer" containerID="ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030" Jul 28 19:39:15.832045 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.832026 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030"} err="failed to get container status \"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030\": rpc error: code = NotFound desc = could not find container \"ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030\": container with ID starting with ee1e977eb30d59118da6312ff816bdb20c1513d043608ff86522f74f8cd5d030 not found: ID does not exist" Jul 28 19:39:15.832045 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.832045 2564 scope.go:117] "RemoveContainer" containerID="5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb" Jul 28 19:39:15.832228 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.832211 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb"} err="failed to get container status \"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb\": rpc error: code = NotFound desc = could not find container \"5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb\": container with ID starting with 5e26d3e097942de6108efde00d210bcd403396fe57cb68618acb668108d650eb not found: ID does not exist" Jul 28 19:39:15.832277 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.832228 2564 scope.go:117] "RemoveContainer" containerID="4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898" Jul 28 19:39:15.832464 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.832434 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898"} err="failed to get container status \"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898\": rpc error: code = NotFound desc = could not find container \"4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898\": container with ID starting with 4d053eb173baade591217d7e81d4fabe89e03fa10b2b9baced0a7389ae5c5898 not found: ID does not exist" Jul 28 19:39:15.832464 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.832448 2564 scope.go:117] "RemoveContainer" containerID="112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b" Jul 28 19:39:15.832675 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.832660 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b"} err="failed to get container status \"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b\": rpc error: code = NotFound desc = could not find container \"112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b\": container with ID starting with 112bae6e0fcebdcfd64d25fc93ad17ef9979fd876ba6520a63912bedd6e5842b not found: ID does not exist" Jul 28 19:39:15.832675 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.832675 2564 scope.go:117] "RemoveContainer" containerID="84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a" Jul 28 19:39:15.832894 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.832873 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a"} err="failed to get container status \"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a\": rpc error: code = NotFound desc = could not find container \"84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a\": container with ID starting with 84c67ca0785a31db4dbb8b567128f2762b092b4b2ca5cce8a1dff22f71a40b3a not found: ID does not exist" Jul 28 19:39:15.879162 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879141 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879162 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879161 2564 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879270 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879171 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879270 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879181 2564 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-kube-rbac-proxy\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879270 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879191 2564 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-metrics-client-certs\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879270 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879200 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-db\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879270 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879208 2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a9613c2-3835-416d-bf81-089120dcca9e-tls-assets\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879270 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879216 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879270 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879224 2564 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879270 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879249 2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a9613c2-3835-416d-bf81-089120dcca9e-config-out\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879270 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879259 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879270 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879266 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879565 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879275 2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-web-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879565 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879285 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879565 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879294 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-configmap-metrics-client-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879565 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879303 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879565 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879310 2564 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a9613c2-3835-416d-bf81-089120dcca9e-secret-grpc-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879565 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879318 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k86sd\" (UniqueName: \"kubernetes.io/projected/4a9613c2-3835-416d-bf81-089120dcca9e-kube-api-access-k86sd\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:15.879565 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:15.879325 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a9613c2-3835-416d-bf81-089120dcca9e-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:16.073852 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.073822 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 19:39:16.078248 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.078223 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 19:39:16.103743 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.103719 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 19:39:16.104002 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.103991 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="config-reloader" Jul 28 19:39:16.104043 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104004 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="config-reloader" Jul 28 19:39:16.104043 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104012 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="kube-rbac-proxy-web" Jul 28 19:39:16.104043 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104017 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="kube-rbac-proxy-web" Jul 28 19:39:16.104043 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104029 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="kube-rbac-proxy" Jul 28 19:39:16.104043 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104034 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="kube-rbac-proxy" Jul 28 19:39:16.104043 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104044 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="init-config-reloader" Jul 28 19:39:16.104201 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104050 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="init-config-reloader" Jul 28 19:39:16.104201 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104058 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="prometheus" Jul 28 19:39:16.104201 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104063 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="prometheus" Jul 28 19:39:16.104201 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104071 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="kube-rbac-proxy-thanos" Jul 28 19:39:16.104201 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104076 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="kube-rbac-proxy-thanos" Jul 28 19:39:16.104201 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104083 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="thanos-sidecar" Jul 28 19:39:16.104201 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104088 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="thanos-sidecar" Jul 28 19:39:16.104201 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104129 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="thanos-sidecar" Jul 28 19:39:16.104201 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104137 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="kube-rbac-proxy-thanos" Jul 28 19:39:16.104201 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104143 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="config-reloader" Jul 28 19:39:16.104201 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104149 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="kube-rbac-proxy-web" Jul 28 19:39:16.104201 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104154 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="kube-rbac-proxy" Jul 28 19:39:16.104201 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.104159 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" containerName="prometheus" Jul 28 19:39:16.107972 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.107951 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.110803 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.110777 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 28 19:39:16.110927 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.110819 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 28 19:39:16.110927 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.110783 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-eq45rqmpcm345\"" Jul 28 19:39:16.111113 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.111075 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 28 19:39:16.111113 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.111100 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-s2r9v\"" Jul 28 19:39:16.111288 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.111112 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 28 19:39:16.111288 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.111124 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 28 19:39:16.111288 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.111131 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 28 19:39:16.111495 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.111446 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 28 19:39:16.111495 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.111463 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 28 19:39:16.111495 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.111471 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 28 19:39:16.111495 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.111474 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 28 19:39:16.111755 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.111504 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 28 19:39:16.111755 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.111451 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 28 19:39:16.114756 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.114737 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 28 19:39:16.116230 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.116211 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 28 19:39:16.122725 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.122705 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 19:39:16.181900 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.181871 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.181900 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.181902 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvqgs\" (UniqueName: \"kubernetes.io/projected/2c8b602a-4310-4a64-8b33-eb45a6929af4-kube-api-access-vvqgs\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182044 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.181928 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182044 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.181958 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182044 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.181980 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182044 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.182001 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c8b602a-4310-4a64-8b33-eb45a6929af4-config-out\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182044 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.182019 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182044 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.182037 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.182055 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-config\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.182074 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.182089 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.182107 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.182122 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.182174 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.182224 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-web-config\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182311 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.182257 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c8b602a-4310-4a64-8b33-eb45a6929af4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182592 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.182317 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182592 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.182344 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182592 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.182379 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.182592 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.182427 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c8b602a-4310-4a64-8b33-eb45a6929af4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.283749 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.283727 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.283873 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.283756 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c8b602a-4310-4a64-8b33-eb45a6929af4-config-out\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.283873 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.283779 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.283873 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.283797 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.284025 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.283908 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-config\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.284025 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.283940 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.284025 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.283964 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.284025 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.283991 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.284025 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.284017 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.284597 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.284538 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.284597 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.284564 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.284793 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.284612 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.284793 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.284615 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.284793 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.284704 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-web-config\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.284793 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.284737 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c8b602a-4310-4a64-8b33-eb45a6929af4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.284793 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.284764 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.284793 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.284790 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.285090 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.284832 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.285090 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.284872 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c8b602a-4310-4a64-8b33-eb45a6929af4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.285090 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.284903 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.285090 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.284918 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.285090 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.284942 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvqgs\" (UniqueName: \"kubernetes.io/projected/2c8b602a-4310-4a64-8b33-eb45a6929af4-kube-api-access-vvqgs\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.285090 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.284980 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.285090 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.285013 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.287066 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.285470 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2c8b602a-4310-4a64-8b33-eb45a6929af4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.287066 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.286311 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.287066 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.286926 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-config\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.287272 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.287130 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.287414 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.287371 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.288136 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.287666 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c8b602a-4310-4a64-8b33-eb45a6929af4-config-out\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.288136 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.287830 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.288136 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.288087 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.288810 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.288781 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2c8b602a-4310-4a64-8b33-eb45a6929af4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.288901 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.288859 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.289083 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.289066 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c8b602a-4310-4a64-8b33-eb45a6929af4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.289386 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.289362 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.289426 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.289403 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-web-config\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.289856 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.289835 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.289933 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.289880 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c8b602a-4310-4a64-8b33-eb45a6929af4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.294084 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.294064 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvqgs\" (UniqueName: \"kubernetes.io/projected/2c8b602a-4310-4a64-8b33-eb45a6929af4-kube-api-access-vvqgs\") pod \"prometheus-k8s-0\" (UID: \"2c8b602a-4310-4a64-8b33-eb45a6929af4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.417909 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.417824 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:16.551908 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.551878 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 19:39:16.554492 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:39:16.554465 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c8b602a_4310_4a64_8b33_eb45a6929af4.slice/crio-ad14496395dc6864e8271ee5b8c267f0308ba24bbdf0e2616e8cc3bebe6d016a WatchSource:0}: Error finding container ad14496395dc6864e8271ee5b8c267f0308ba24bbdf0e2616e8cc3bebe6d016a: Status 404 returned error can't find the container with id ad14496395dc6864e8271ee5b8c267f0308ba24bbdf0e2616e8cc3bebe6d016a Jul 28 19:39:16.753429 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.753357 2564 generic.go:356] "Generic (PLEG): container finished" podID="2c8b602a-4310-4a64-8b33-eb45a6929af4" containerID="ddc5fcfdd8b6400e0b03599a67c1f4b21bc890611b9a571845b67c40f8e894af" exitCode=0 Jul 28 19:39:16.753429 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.753421 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c8b602a-4310-4a64-8b33-eb45a6929af4","Type":"ContainerDied","Data":"ddc5fcfdd8b6400e0b03599a67c1f4b21bc890611b9a571845b67c40f8e894af"} Jul 28 19:39:16.753803 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.753478 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c8b602a-4310-4a64-8b33-eb45a6929af4","Type":"ContainerStarted","Data":"ad14496395dc6864e8271ee5b8c267f0308ba24bbdf0e2616e8cc3bebe6d016a"} Jul 28 19:39:16.919746 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:16.919716 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a9613c2-3835-416d-bf81-089120dcca9e" path="/var/lib/kubelet/pods/4a9613c2-3835-416d-bf81-089120dcca9e/volumes" Jul 28 19:39:17.761803 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:17.761768 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c8b602a-4310-4a64-8b33-eb45a6929af4","Type":"ContainerStarted","Data":"7347c0ad75ac6747af46153cfc00e3d649b4aee0420d7629ab54d0034378c0af"} Jul 28 19:39:17.762192 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:17.761833 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c8b602a-4310-4a64-8b33-eb45a6929af4","Type":"ContainerStarted","Data":"5209e3dd37fe746c809cd08a0689fa9a4af2084bb58dc5bf0005703c33fc72b3"} Jul 28 19:39:17.762192 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:17.761860 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c8b602a-4310-4a64-8b33-eb45a6929af4","Type":"ContainerStarted","Data":"8effb7aaa4a8f2c4307f9976098164d964081f8fb6da75cda01fe9e8d433671d"} Jul 28 19:39:17.762192 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:17.761890 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c8b602a-4310-4a64-8b33-eb45a6929af4","Type":"ContainerStarted","Data":"412182b343ace5645afe141ee1d1b561bbf9f350c3ebddeced077bcee4c915a2"} Jul 28 19:39:17.762192 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:17.761915 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c8b602a-4310-4a64-8b33-eb45a6929af4","Type":"ContainerStarted","Data":"13847974863faebee5271b9fc87eaa539f47a05dbbe5aec9e3a771d9c2ba0400"} Jul 28 19:39:17.762192 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:17.761936 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2c8b602a-4310-4a64-8b33-eb45a6929af4","Type":"ContainerStarted","Data":"8a96299ce18ea3b2e78b4763093f4d202f36dd32b0a67cf23d93546b085b5904"} Jul 28 19:39:17.792846 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:17.792801 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.77349366 podStartE2EDuration="1.792785627s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.291967ms" isStatefulPod=false podCreationTimestamp="2026-07-28 19:39:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:39:17.791538703 +0000 UTC m=+153.498750631" watchObservedRunningTime="2026-07-28 19:39:17.792785627 +0000 UTC m=+153.499997519" Jul 28 19:39:19.857951 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:19.857916 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85f554765c-ffl4h"] Jul 28 19:39:19.860971 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:19.860950 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:19.870302 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:19.870275 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85f554765c-ffl4h"] Jul 28 19:39:19.918711 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:19.918683 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-trusted-ca-bundle\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:19.918833 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:19.918724 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxjmx\" (UniqueName: \"kubernetes.io/projected/b3ef2308-9c31-4d59-b7f2-80b71963fb50-kube-api-access-rxjmx\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:19.918833 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:19.918745 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-oauth-serving-cert\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:19.918833 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:19.918821 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-oauth-config\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:19.918946 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:19.918853 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-config\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:19.918946 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:19.918871 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-serving-cert\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:19.918946 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:19.918904 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-service-ca\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.020023 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.019986 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxjmx\" (UniqueName: \"kubernetes.io/projected/b3ef2308-9c31-4d59-b7f2-80b71963fb50-kube-api-access-rxjmx\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.020023 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.020025 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-oauth-serving-cert\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.020216 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.020051 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-oauth-config\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.020216 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.020071 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-config\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.020216 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.020096 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-serving-cert\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.020216 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.020124 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-service-ca\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.020216 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.020192 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-trusted-ca-bundle\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.020875 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.020848 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-oauth-serving-cert\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.020986 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.020918 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-service-ca\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.021044 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.020978 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-config\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.021217 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.021197 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-trusted-ca-bundle\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.022570 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.022547 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-serving-cert\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.022673 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.022574 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-oauth-config\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.028925 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.028904 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxjmx\" (UniqueName: \"kubernetes.io/projected/b3ef2308-9c31-4d59-b7f2-80b71963fb50-kube-api-access-rxjmx\") pod \"console-85f554765c-ffl4h\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.170774 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.170671 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:20.298299 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.298264 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85f554765c-ffl4h"] Jul 28 19:39:20.300283 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:39:20.300252 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ef2308_9c31_4d59_b7f2_80b71963fb50.slice/crio-f33cb9700a4510cf42aa44106620d12a035d2114fd6ffa9b24e547decedf3598 WatchSource:0}: Error finding container f33cb9700a4510cf42aa44106620d12a035d2114fd6ffa9b24e547decedf3598: Status 404 returned error can't find the container with id f33cb9700a4510cf42aa44106620d12a035d2114fd6ffa9b24e547decedf3598 Jul 28 19:39:20.776510 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.776477 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f554765c-ffl4h" event={"ID":"b3ef2308-9c31-4d59-b7f2-80b71963fb50","Type":"ContainerStarted","Data":"e72cc3bce151cc7366be1797353ae19d3bd4065cf9808c85ce20457ad63dac3c"} Jul 28 19:39:20.776676 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.776529 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f554765c-ffl4h" event={"ID":"b3ef2308-9c31-4d59-b7f2-80b71963fb50","Type":"ContainerStarted","Data":"f33cb9700a4510cf42aa44106620d12a035d2114fd6ffa9b24e547decedf3598"} Jul 28 19:39:20.796397 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:20.796346 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85f554765c-ffl4h" podStartSLOduration=1.796328204 podStartE2EDuration="1.796328204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:39:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:39:20.794456283 +0000 UTC m=+156.501668176" watchObservedRunningTime="2026-07-28 19:39:20.796328204 +0000 UTC m=+156.503540095" Jul 28 19:39:21.418477 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:21.418430 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:39:30.171292 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:30.171254 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:30.171681 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:30.171313 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:30.176046 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:30.176023 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:30.826561 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:30.826534 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:39:30.876188 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:30.876144 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65964b56b7-brxdg"] Jul 28 19:39:55.900736 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:55.900678 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65964b56b7-brxdg" podUID="a66ca1c5-3eb8-40d8-a240-c61694a7ca6e" containerName="console" containerID="cri-o://a6296f31edab126bf43ab1849972c5b0bf4831f0de2bf629453b638def19ade2" gracePeriod=15 Jul 28 19:39:56.137283 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.137262 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:39:56.244504 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.244418 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-oauth-serving-cert\") pod \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " Jul 28 19:39:56.244504 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.244458 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbkfw\" (UniqueName: \"kubernetes.io/projected/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-kube-api-access-gbkfw\") pod \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " Jul 28 19:39:56.244504 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.244492 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-oauth-config\") pod \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " Jul 28 19:39:56.244789 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.244603 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-trusted-ca-bundle\") pod \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " Jul 28 19:39:56.244789 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.244762 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-service-ca\") pod \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " Jul 28 19:39:56.244900 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.244814 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-serving-cert\") pod \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " Jul 28 19:39:56.244900 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.244837 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-config\") pod \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\" (UID: \"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e\") " Jul 28 19:39:56.245000 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.244914 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a66ca1c5-3eb8-40d8-a240-c61694a7ca6e" (UID: "a66ca1c5-3eb8-40d8-a240-c61694a7ca6e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:39:56.245000 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.244976 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a66ca1c5-3eb8-40d8-a240-c61694a7ca6e" (UID: "a66ca1c5-3eb8-40d8-a240-c61694a7ca6e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:39:56.245154 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.245133 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-oauth-serving-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:56.245154 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.245142 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-service-ca" (OuterVolumeSpecName: "service-ca") pod "a66ca1c5-3eb8-40d8-a240-c61694a7ca6e" (UID: "a66ca1c5-3eb8-40d8-a240-c61694a7ca6e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:39:56.245154 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.245159 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-trusted-ca-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:56.245387 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.245272 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-config" (OuterVolumeSpecName: "console-config") pod "a66ca1c5-3eb8-40d8-a240-c61694a7ca6e" (UID: "a66ca1c5-3eb8-40d8-a240-c61694a7ca6e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:39:56.246672 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.246647 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a66ca1c5-3eb8-40d8-a240-c61694a7ca6e" (UID: "a66ca1c5-3eb8-40d8-a240-c61694a7ca6e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:56.247140 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.247115 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a66ca1c5-3eb8-40d8-a240-c61694a7ca6e" (UID: "a66ca1c5-3eb8-40d8-a240-c61694a7ca6e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:39:56.247140 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.247126 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-kube-api-access-gbkfw" (OuterVolumeSpecName: "kube-api-access-gbkfw") pod "a66ca1c5-3eb8-40d8-a240-c61694a7ca6e" (UID: "a66ca1c5-3eb8-40d8-a240-c61694a7ca6e"). InnerVolumeSpecName "kube-api-access-gbkfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 19:39:56.346376 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.346337 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbkfw\" (UniqueName: \"kubernetes.io/projected/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-kube-api-access-gbkfw\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:56.346376 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.346370 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-oauth-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:56.346376 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.346379 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:56.346376 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.346387 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-serving-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:56.346628 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.346396 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e-console-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:39:56.931009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.930979 2564 generic.go:356] "Generic (PLEG): container finished" podID="a66ca1c5-3eb8-40d8-a240-c61694a7ca6e" containerID="a6296f31edab126bf43ab1849972c5b0bf4831f0de2bf629453b638def19ade2" exitCode=0 Jul 28 19:39:56.931410 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.931035 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65964b56b7-brxdg" Jul 28 19:39:56.931410 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.931069 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65964b56b7-brxdg" event={"ID":"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e","Type":"ContainerDied","Data":"a6296f31edab126bf43ab1849972c5b0bf4831f0de2bf629453b638def19ade2"} Jul 28 19:39:56.931410 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.931133 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65964b56b7-brxdg" event={"ID":"a66ca1c5-3eb8-40d8-a240-c61694a7ca6e","Type":"ContainerDied","Data":"e2182508d41680b5d35d520838526fe02414994a5c01e2a71945fa6da42f209e"} Jul 28 19:39:56.931410 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.931162 2564 scope.go:117] "RemoveContainer" containerID="a6296f31edab126bf43ab1849972c5b0bf4831f0de2bf629453b638def19ade2" Jul 28 19:39:56.939095 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.939079 2564 scope.go:117] "RemoveContainer" containerID="a6296f31edab126bf43ab1849972c5b0bf4831f0de2bf629453b638def19ade2" Jul 28 19:39:56.939354 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:39:56.939337 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6296f31edab126bf43ab1849972c5b0bf4831f0de2bf629453b638def19ade2\": container with ID starting with a6296f31edab126bf43ab1849972c5b0bf4831f0de2bf629453b638def19ade2 not found: ID does not exist" containerID="a6296f31edab126bf43ab1849972c5b0bf4831f0de2bf629453b638def19ade2" Jul 28 19:39:56.939401 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.939361 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6296f31edab126bf43ab1849972c5b0bf4831f0de2bf629453b638def19ade2"} err="failed to get container status \"a6296f31edab126bf43ab1849972c5b0bf4831f0de2bf629453b638def19ade2\": rpc error: code = NotFound desc = could not find container \"a6296f31edab126bf43ab1849972c5b0bf4831f0de2bf629453b638def19ade2\": container with ID starting with a6296f31edab126bf43ab1849972c5b0bf4831f0de2bf629453b638def19ade2 not found: ID does not exist" Jul 28 19:39:56.952509 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.952485 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65964b56b7-brxdg"] Jul 28 19:39:56.957758 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:56.957730 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65964b56b7-brxdg"] Jul 28 19:39:58.918604 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:39:58.918576 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a66ca1c5-3eb8-40d8-a240-c61694a7ca6e" path="/var/lib/kubelet/pods/a66ca1c5-3eb8-40d8-a240-c61694a7ca6e/volumes" Jul 28 19:40:16.418717 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:16.418681 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:40:16.434130 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:16.434106 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:40:17.029507 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:17.029474 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 19:40:29.801539 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:29.801509 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7ff495695f-gq5md"] Jul 28 19:40:29.802009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:29.801872 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a66ca1c5-3eb8-40d8-a240-c61694a7ca6e" containerName="console" Jul 28 19:40:29.802009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:29.801885 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a66ca1c5-3eb8-40d8-a240-c61694a7ca6e" containerName="console" Jul 28 19:40:29.802009 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:29.801969 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a66ca1c5-3eb8-40d8-a240-c61694a7ca6e" containerName="console" Jul 28 19:40:29.806158 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:29.806136 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:29.818514 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:29.818493 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ff495695f-gq5md"] Jul 28 19:40:29.929570 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:29.929539 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-trusted-ca-bundle\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:29.929570 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:29.929572 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dlg6\" (UniqueName: \"kubernetes.io/projected/1df4aefd-3e9b-4065-aa4e-38165545bdab-kube-api-access-6dlg6\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:29.929813 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:29.929594 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-oauth-config\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:29.929813 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:29.929716 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-serving-cert\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:29.929813 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:29.929745 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-oauth-serving-cert\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:29.929813 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:29.929764 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-service-ca\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:29.929939 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:29.929856 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-config\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.030617 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.030586 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-config\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.030743 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.030681 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-trusted-ca-bundle\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.030743 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.030705 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dlg6\" (UniqueName: \"kubernetes.io/projected/1df4aefd-3e9b-4065-aa4e-38165545bdab-kube-api-access-6dlg6\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.030743 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.030733 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-oauth-config\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.030890 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.030783 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-serving-cert\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.030890 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.030810 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-oauth-serving-cert\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.030890 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.030837 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-service-ca\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.031459 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.031429 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-config\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.031603 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.031510 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-service-ca\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.031603 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.031508 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-oauth-serving-cert\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.031603 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.031559 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-trusted-ca-bundle\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.033404 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.033370 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-serving-cert\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.033514 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.033403 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-oauth-config\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.041349 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.041329 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dlg6\" (UniqueName: \"kubernetes.io/projected/1df4aefd-3e9b-4065-aa4e-38165545bdab-kube-api-access-6dlg6\") pod \"console-7ff495695f-gq5md\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.115549 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.115463 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:30.246546 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:30.246524 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ff495695f-gq5md"] Jul 28 19:40:30.248628 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:40:30.248602 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1df4aefd_3e9b_4065_aa4e_38165545bdab.slice/crio-04596b0f74a9421102373f77bcf77ab2fcc118b88223aa55a9b9cbc8cb8cbb10 WatchSource:0}: Error finding container 04596b0f74a9421102373f77bcf77ab2fcc118b88223aa55a9b9cbc8cb8cbb10: Status 404 returned error can't find the container with id 04596b0f74a9421102373f77bcf77ab2fcc118b88223aa55a9b9cbc8cb8cbb10 Jul 28 19:40:31.069608 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:31.069577 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff495695f-gq5md" event={"ID":"1df4aefd-3e9b-4065-aa4e-38165545bdab","Type":"ContainerStarted","Data":"21c8b9c659960ef1c4f03933e79730cbcbdb1892ca6c97c2d2c10740f8751d71"} Jul 28 19:40:31.070037 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:31.069622 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff495695f-gq5md" event={"ID":"1df4aefd-3e9b-4065-aa4e-38165545bdab","Type":"ContainerStarted","Data":"04596b0f74a9421102373f77bcf77ab2fcc118b88223aa55a9b9cbc8cb8cbb10"} Jul 28 19:40:31.088883 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:31.088840 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7ff495695f-gq5md" podStartSLOduration=2.088821054 podStartE2EDuration="2.088821054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:40:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:40:31.088305031 +0000 UTC m=+226.795516930" watchObservedRunningTime="2026-07-28 19:40:31.088821054 +0000 UTC m=+226.796032947" Jul 28 19:40:40.115823 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:40.115780 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:40.116211 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:40.115874 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:40.120585 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:40.120560 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:41.119008 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:41.118982 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:40:41.199765 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:40:41.199732 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85f554765c-ffl4h"] Jul 28 19:41:06.219014 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.218924 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85f554765c-ffl4h" podUID="b3ef2308-9c31-4d59-b7f2-80b71963fb50" containerName="console" containerID="cri-o://e72cc3bce151cc7366be1797353ae19d3bd4065cf9808c85ce20457ad63dac3c" gracePeriod=15 Jul 28 19:41:06.465577 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.465549 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:41:06.540324 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.540250 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-service-ca\") pod \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " Jul 28 19:41:06.540324 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.540307 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-serving-cert\") pod \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " Jul 28 19:41:06.540556 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.540339 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxjmx\" (UniqueName: \"kubernetes.io/projected/b3ef2308-9c31-4d59-b7f2-80b71963fb50-kube-api-access-rxjmx\") pod \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " Jul 28 19:41:06.540556 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.540435 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-oauth-config\") pod \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " Jul 28 19:41:06.540556 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.540479 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-config\") pod \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " Jul 28 19:41:06.540556 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.540509 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-oauth-serving-cert\") pod \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " Jul 28 19:41:06.540793 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.540559 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-trusted-ca-bundle\") pod \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\" (UID: \"b3ef2308-9c31-4d59-b7f2-80b71963fb50\") " Jul 28 19:41:06.540793 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.540706 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-service-ca" (OuterVolumeSpecName: "service-ca") pod "b3ef2308-9c31-4d59-b7f2-80b71963fb50" (UID: "b3ef2308-9c31-4d59-b7f2-80b71963fb50"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:41:06.540911 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.540883 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-config" (OuterVolumeSpecName: "console-config") pod "b3ef2308-9c31-4d59-b7f2-80b71963fb50" (UID: "b3ef2308-9c31-4d59-b7f2-80b71963fb50"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:41:06.540974 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.540950 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b3ef2308-9c31-4d59-b7f2-80b71963fb50" (UID: "b3ef2308-9c31-4d59-b7f2-80b71963fb50"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:41:06.541024 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.540995 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:41:06.541024 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.541014 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:41:06.541024 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.541016 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b3ef2308-9c31-4d59-b7f2-80b71963fb50" (UID: "b3ef2308-9c31-4d59-b7f2-80b71963fb50"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:41:06.542547 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.542516 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b3ef2308-9c31-4d59-b7f2-80b71963fb50" (UID: "b3ef2308-9c31-4d59-b7f2-80b71963fb50"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:41:06.542656 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.542549 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b3ef2308-9c31-4d59-b7f2-80b71963fb50" (UID: "b3ef2308-9c31-4d59-b7f2-80b71963fb50"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:41:06.542656 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.542568 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3ef2308-9c31-4d59-b7f2-80b71963fb50-kube-api-access-rxjmx" (OuterVolumeSpecName: "kube-api-access-rxjmx") pod "b3ef2308-9c31-4d59-b7f2-80b71963fb50" (UID: "b3ef2308-9c31-4d59-b7f2-80b71963fb50"). InnerVolumeSpecName "kube-api-access-rxjmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 19:41:06.647192 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.642929 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-trusted-ca-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:41:06.647192 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.642968 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-serving-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:41:06.647192 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.642984 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxjmx\" (UniqueName: \"kubernetes.io/projected/b3ef2308-9c31-4d59-b7f2-80b71963fb50-kube-api-access-rxjmx\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:41:06.647192 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.642998 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3ef2308-9c31-4d59-b7f2-80b71963fb50-console-oauth-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:41:06.647192 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:06.643013 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3ef2308-9c31-4d59-b7f2-80b71963fb50-oauth-serving-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:41:07.225130 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:07.225038 2564 generic.go:356] "Generic (PLEG): container finished" podID="b3ef2308-9c31-4d59-b7f2-80b71963fb50" containerID="e72cc3bce151cc7366be1797353ae19d3bd4065cf9808c85ce20457ad63dac3c" exitCode=0 Jul 28 19:41:07.225130 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:07.225100 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f554765c-ffl4h" Jul 28 19:41:07.225587 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:07.225123 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f554765c-ffl4h" event={"ID":"b3ef2308-9c31-4d59-b7f2-80b71963fb50","Type":"ContainerDied","Data":"e72cc3bce151cc7366be1797353ae19d3bd4065cf9808c85ce20457ad63dac3c"} Jul 28 19:41:07.225587 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:07.225174 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f554765c-ffl4h" event={"ID":"b3ef2308-9c31-4d59-b7f2-80b71963fb50","Type":"ContainerDied","Data":"f33cb9700a4510cf42aa44106620d12a035d2114fd6ffa9b24e547decedf3598"} Jul 28 19:41:07.225587 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:07.225195 2564 scope.go:117] "RemoveContainer" containerID="e72cc3bce151cc7366be1797353ae19d3bd4065cf9808c85ce20457ad63dac3c" Jul 28 19:41:07.233383 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:07.233367 2564 scope.go:117] "RemoveContainer" containerID="e72cc3bce151cc7366be1797353ae19d3bd4065cf9808c85ce20457ad63dac3c" Jul 28 19:41:07.233686 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:41:07.233663 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e72cc3bce151cc7366be1797353ae19d3bd4065cf9808c85ce20457ad63dac3c\": container with ID starting with e72cc3bce151cc7366be1797353ae19d3bd4065cf9808c85ce20457ad63dac3c not found: ID does not exist" containerID="e72cc3bce151cc7366be1797353ae19d3bd4065cf9808c85ce20457ad63dac3c" Jul 28 19:41:07.233787 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:07.233695 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e72cc3bce151cc7366be1797353ae19d3bd4065cf9808c85ce20457ad63dac3c"} err="failed to get container status \"e72cc3bce151cc7366be1797353ae19d3bd4065cf9808c85ce20457ad63dac3c\": rpc error: code = NotFound desc = could not find container \"e72cc3bce151cc7366be1797353ae19d3bd4065cf9808c85ce20457ad63dac3c\": container with ID starting with e72cc3bce151cc7366be1797353ae19d3bd4065cf9808c85ce20457ad63dac3c not found: ID does not exist" Jul 28 19:41:07.243249 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:07.243230 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85f554765c-ffl4h"] Jul 28 19:41:07.246275 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:07.246254 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85f554765c-ffl4h"] Jul 28 19:41:08.917502 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:08.917462 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3ef2308-9c31-4d59-b7f2-80b71963fb50" path="/var/lib/kubelet/pods/b3ef2308-9c31-4d59-b7f2-80b71963fb50/volumes" Jul 28 19:41:44.729506 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:44.729462 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 28 19:41:52.440251 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.440218 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-26snj"] Jul 28 19:41:52.442419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.440530 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3ef2308-9c31-4d59-b7f2-80b71963fb50" containerName="console" Jul 28 19:41:52.442419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.440540 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3ef2308-9c31-4d59-b7f2-80b71963fb50" containerName="console" Jul 28 19:41:52.442419 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.440613 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3ef2308-9c31-4d59-b7f2-80b71963fb50" containerName="console" Jul 28 19:41:52.443354 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.443340 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-26snj" Jul 28 19:41:52.446144 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.446124 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 28 19:41:52.446249 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.446147 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 28 19:41:52.446974 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.446958 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-cr6bb\"" Jul 28 19:41:52.453122 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.453101 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-26snj"] Jul 28 19:41:52.532907 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.532869 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/072a0bd7-b31a-4aa3-a7ce-cd9a6d5969ae-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-26snj\" (UID: \"072a0bd7-b31a-4aa3-a7ce-cd9a6d5969ae\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-26snj" Jul 28 19:41:52.533086 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.532913 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wspzv\" (UniqueName: \"kubernetes.io/projected/072a0bd7-b31a-4aa3-a7ce-cd9a6d5969ae-kube-api-access-wspzv\") pod \"cert-manager-cainjector-559cdd8c87-26snj\" (UID: \"072a0bd7-b31a-4aa3-a7ce-cd9a6d5969ae\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-26snj" Jul 28 19:41:52.633283 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.633248 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/072a0bd7-b31a-4aa3-a7ce-cd9a6d5969ae-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-26snj\" (UID: \"072a0bd7-b31a-4aa3-a7ce-cd9a6d5969ae\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-26snj" Jul 28 19:41:52.633438 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.633291 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wspzv\" (UniqueName: \"kubernetes.io/projected/072a0bd7-b31a-4aa3-a7ce-cd9a6d5969ae-kube-api-access-wspzv\") pod \"cert-manager-cainjector-559cdd8c87-26snj\" (UID: \"072a0bd7-b31a-4aa3-a7ce-cd9a6d5969ae\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-26snj" Jul 28 19:41:52.642761 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.642734 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/072a0bd7-b31a-4aa3-a7ce-cd9a6d5969ae-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-26snj\" (UID: \"072a0bd7-b31a-4aa3-a7ce-cd9a6d5969ae\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-26snj" Jul 28 19:41:52.642890 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.642811 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wspzv\" (UniqueName: \"kubernetes.io/projected/072a0bd7-b31a-4aa3-a7ce-cd9a6d5969ae-kube-api-access-wspzv\") pod \"cert-manager-cainjector-559cdd8c87-26snj\" (UID: \"072a0bd7-b31a-4aa3-a7ce-cd9a6d5969ae\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-26snj" Jul 28 19:41:52.761259 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.761176 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-26snj" Jul 28 19:41:52.889550 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.889511 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-26snj"] Jul 28 19:41:52.897844 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:41:52.897816 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod072a0bd7_b31a_4aa3_a7ce_cd9a6d5969ae.slice/crio-e686ea81e2a70419ae01397a359522790c5152dcd6b057005d259c93ce986acf WatchSource:0}: Error finding container e686ea81e2a70419ae01397a359522790c5152dcd6b057005d259c93ce986acf: Status 404 returned error can't find the container with id e686ea81e2a70419ae01397a359522790c5152dcd6b057005d259c93ce986acf Jul 28 19:41:52.899625 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:52.899609 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 19:41:53.415217 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:53.415179 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-26snj" event={"ID":"072a0bd7-b31a-4aa3-a7ce-cd9a6d5969ae","Type":"ContainerStarted","Data":"e686ea81e2a70419ae01397a359522790c5152dcd6b057005d259c93ce986acf"} Jul 28 19:41:57.436793 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:57.436757 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-26snj" event={"ID":"072a0bd7-b31a-4aa3-a7ce-cd9a6d5969ae","Type":"ContainerStarted","Data":"7c384cc9fe642454afab4b78b303ec23137d688fac7665421d40014d6eb51ae0"} Jul 28 19:41:57.454358 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:41:57.454315 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-26snj" podStartSLOduration=1.482785429 podStartE2EDuration="5.454301451s" totalImagesPullingTime="3.971516022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:41:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:41:57.452685658 +0000 UTC m=+313.159897552" watchObservedRunningTime="2026-07-28 19:41:57.454301451 +0000 UTC m=+313.161513344" Jul 28 19:42:02.647890 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:02.647852 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6slld"] Jul 28 19:42:02.651382 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:02.651361 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6slld" Jul 28 19:42:02.654220 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:02.654192 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 28 19:42:02.654346 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:02.654323 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 28 19:42:02.655332 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:02.655316 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-c6ltj\"" Jul 28 19:42:02.659145 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:02.659126 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6slld"] Jul 28 19:42:02.820744 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:02.820707 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndbzs\" (UniqueName: \"kubernetes.io/projected/4d428e18-6930-4408-aa27-734824a79dbf-kube-api-access-ndbzs\") pod \"openshift-lws-operator-fd8ccff4c-6slld\" (UID: \"4d428e18-6930-4408-aa27-734824a79dbf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6slld" Jul 28 19:42:02.820952 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:02.820801 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d428e18-6930-4408-aa27-734824a79dbf-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6slld\" (UID: \"4d428e18-6930-4408-aa27-734824a79dbf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6slld" Jul 28 19:42:02.921835 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:02.921750 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndbzs\" (UniqueName: \"kubernetes.io/projected/4d428e18-6930-4408-aa27-734824a79dbf-kube-api-access-ndbzs\") pod \"openshift-lws-operator-fd8ccff4c-6slld\" (UID: \"4d428e18-6930-4408-aa27-734824a79dbf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6slld" Jul 28 19:42:02.921835 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:02.921814 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d428e18-6930-4408-aa27-734824a79dbf-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6slld\" (UID: \"4d428e18-6930-4408-aa27-734824a79dbf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6slld" Jul 28 19:42:02.922134 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:02.922119 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d428e18-6930-4408-aa27-734824a79dbf-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6slld\" (UID: \"4d428e18-6930-4408-aa27-734824a79dbf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6slld" Jul 28 19:42:02.934840 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:02.934814 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndbzs\" (UniqueName: \"kubernetes.io/projected/4d428e18-6930-4408-aa27-734824a79dbf-kube-api-access-ndbzs\") pod \"openshift-lws-operator-fd8ccff4c-6slld\" (UID: \"4d428e18-6930-4408-aa27-734824a79dbf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6slld" Jul 28 19:42:02.961416 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:02.961386 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6slld" Jul 28 19:42:03.096772 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:03.096745 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6slld"] Jul 28 19:42:03.099211 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:42:03.099184 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d428e18_6930_4408_aa27_734824a79dbf.slice/crio-ce331068591209bf0fab48cb4760a99751d1844aa246cc1decc8c3beebe0804d WatchSource:0}: Error finding container ce331068591209bf0fab48cb4760a99751d1844aa246cc1decc8c3beebe0804d: Status 404 returned error can't find the container with id ce331068591209bf0fab48cb4760a99751d1844aa246cc1decc8c3beebe0804d Jul 28 19:42:03.461020 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:03.460986 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6slld" event={"ID":"4d428e18-6930-4408-aa27-734824a79dbf","Type":"ContainerStarted","Data":"ce331068591209bf0fab48cb4760a99751d1844aa246cc1decc8c3beebe0804d"} Jul 28 19:42:06.477275 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:06.477242 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6slld" event={"ID":"4d428e18-6930-4408-aa27-734824a79dbf","Type":"ContainerStarted","Data":"39f2904d5a7baeca1f590defe15a5c697ea3c096057aa773d2e943ed095db4e9"} Jul 28 19:42:06.494432 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:06.494381 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6slld" podStartSLOduration=1.709520522 podStartE2EDuration="4.494360765s" totalImagesPullingTime="2.784840243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:06.493756092 +0000 UTC m=+322.200967985" watchObservedRunningTime="2026-07-28 19:42:06.494360765 +0000 UTC m=+322.201572659" Jul 28 19:42:22.081161 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.081127 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt"] Jul 28 19:42:22.084686 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.084669 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.092974 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.092955 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 28 19:42:22.093129 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.093032 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 28 19:42:22.093305 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.093288 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 28 19:42:22.093393 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.093379 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-25r4d\"" Jul 28 19:42:22.096190 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.096173 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 28 19:42:22.114987 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.114954 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt"] Jul 28 19:42:22.184826 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.184785 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6023a46c-0039-4880-a083-7a3fe957b7c6-tmp\") pod \"opendatahub-operator-controller-manager-856746f77b-gl2jt\" (UID: \"6023a46c-0039-4880-a083-7a3fe957b7c6\") " pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.184826 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.184824 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6023a46c-0039-4880-a083-7a3fe957b7c6-manifests\") pod \"opendatahub-operator-controller-manager-856746f77b-gl2jt\" (UID: \"6023a46c-0039-4880-a083-7a3fe957b7c6\") " pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.185033 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.184853 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6023a46c-0039-4880-a083-7a3fe957b7c6-webhook-cert\") pod \"opendatahub-operator-controller-manager-856746f77b-gl2jt\" (UID: \"6023a46c-0039-4880-a083-7a3fe957b7c6\") " pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.185033 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.184873 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqdl4\" (UniqueName: \"kubernetes.io/projected/6023a46c-0039-4880-a083-7a3fe957b7c6-kube-api-access-bqdl4\") pod \"opendatahub-operator-controller-manager-856746f77b-gl2jt\" (UID: \"6023a46c-0039-4880-a083-7a3fe957b7c6\") " pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.185033 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.184947 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6023a46c-0039-4880-a083-7a3fe957b7c6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-856746f77b-gl2jt\" (UID: \"6023a46c-0039-4880-a083-7a3fe957b7c6\") " pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.285819 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.285777 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6023a46c-0039-4880-a083-7a3fe957b7c6-tmp\") pod \"opendatahub-operator-controller-manager-856746f77b-gl2jt\" (UID: \"6023a46c-0039-4880-a083-7a3fe957b7c6\") " pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.285819 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.285819 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6023a46c-0039-4880-a083-7a3fe957b7c6-manifests\") pod \"opendatahub-operator-controller-manager-856746f77b-gl2jt\" (UID: \"6023a46c-0039-4880-a083-7a3fe957b7c6\") " pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.286069 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.285848 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6023a46c-0039-4880-a083-7a3fe957b7c6-webhook-cert\") pod \"opendatahub-operator-controller-manager-856746f77b-gl2jt\" (UID: \"6023a46c-0039-4880-a083-7a3fe957b7c6\") " pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.286069 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.285880 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqdl4\" (UniqueName: \"kubernetes.io/projected/6023a46c-0039-4880-a083-7a3fe957b7c6-kube-api-access-bqdl4\") pod \"opendatahub-operator-controller-manager-856746f77b-gl2jt\" (UID: \"6023a46c-0039-4880-a083-7a3fe957b7c6\") " pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.286069 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.286039 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6023a46c-0039-4880-a083-7a3fe957b7c6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-856746f77b-gl2jt\" (UID: \"6023a46c-0039-4880-a083-7a3fe957b7c6\") " pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.286256 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.286234 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6023a46c-0039-4880-a083-7a3fe957b7c6-manifests\") pod \"opendatahub-operator-controller-manager-856746f77b-gl2jt\" (UID: \"6023a46c-0039-4880-a083-7a3fe957b7c6\") " pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.288288 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.288258 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6023a46c-0039-4880-a083-7a3fe957b7c6-tmp\") pod \"opendatahub-operator-controller-manager-856746f77b-gl2jt\" (UID: \"6023a46c-0039-4880-a083-7a3fe957b7c6\") " pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.288451 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.288434 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6023a46c-0039-4880-a083-7a3fe957b7c6-webhook-cert\") pod \"opendatahub-operator-controller-manager-856746f77b-gl2jt\" (UID: \"6023a46c-0039-4880-a083-7a3fe957b7c6\") " pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.288521 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.288439 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6023a46c-0039-4880-a083-7a3fe957b7c6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-856746f77b-gl2jt\" (UID: \"6023a46c-0039-4880-a083-7a3fe957b7c6\") " pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.298645 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.298612 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqdl4\" (UniqueName: \"kubernetes.io/projected/6023a46c-0039-4880-a083-7a3fe957b7c6-kube-api-access-bqdl4\") pod \"opendatahub-operator-controller-manager-856746f77b-gl2jt\" (UID: \"6023a46c-0039-4880-a083-7a3fe957b7c6\") " pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.394416 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.394318 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:22.573081 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:22.573054 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt"] Jul 28 19:42:22.573707 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:42:22.573684 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6023a46c_0039_4880_a083_7a3fe957b7c6.slice/crio-ea7504f5d3c88a98796a4ca709ed511b1b50cc9a332afd4d767487be80df7917 WatchSource:0}: Error finding container ea7504f5d3c88a98796a4ca709ed511b1b50cc9a332afd4d767487be80df7917: Status 404 returned error can't find the container with id ea7504f5d3c88a98796a4ca709ed511b1b50cc9a332afd4d767487be80df7917 Jul 28 19:42:23.551396 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:23.551351 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" event={"ID":"6023a46c-0039-4880-a083-7a3fe957b7c6","Type":"ContainerStarted","Data":"ea7504f5d3c88a98796a4ca709ed511b1b50cc9a332afd4d767487be80df7917"} Jul 28 19:42:26.571090 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:26.571048 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" event={"ID":"6023a46c-0039-4880-a083-7a3fe957b7c6","Type":"ContainerStarted","Data":"93f6f394bb1d76eeaea5feb721a188b0f72319de911e8043cf807a4035f6a435"} Jul 28 19:42:27.576803 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:27.576767 2564 generic.go:356] "Generic (PLEG): container finished" podID="6023a46c-0039-4880-a083-7a3fe957b7c6" containerID="93f6f394bb1d76eeaea5feb721a188b0f72319de911e8043cf807a4035f6a435" exitCode=0 Jul 28 19:42:27.577274 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:27.576852 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" event={"ID":"6023a46c-0039-4880-a083-7a3fe957b7c6","Type":"ContainerDied","Data":"93f6f394bb1d76eeaea5feb721a188b0f72319de911e8043cf807a4035f6a435"} Jul 28 19:42:28.583071 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:28.583031 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" event={"ID":"6023a46c-0039-4880-a083-7a3fe957b7c6","Type":"ContainerStarted","Data":"98d83b46f8dd03965eeedf469e1a33e3c1a60f57e151ad341d6ef267b2e42c22"} Jul 28 19:42:28.583544 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:28.583168 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:39.589157 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:39.589077 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" Jul 28 19:42:39.614196 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:39.614150 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-856746f77b-gl2jt" podStartSLOduration=13.23569104 podStartE2EDuration="17.614137249s" totalImagesPullingTime="3.332387491s" totalInitContainerRuntime="1.046058718s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:28.625610849 +0000 UTC m=+344.332822743" watchObservedRunningTime="2026-07-28 19:42:39.614137249 +0000 UTC m=+355.321349142" Jul 28 19:42:42.867448 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:42.867409 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp"] Jul 28 19:42:42.870766 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:42.870750 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:42.880146 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:42.880122 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5sxt8\"" Jul 28 19:42:42.880282 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:42.880182 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 28 19:42:42.880282 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:42.880199 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 28 19:42:42.880282 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:42.880201 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 28 19:42:42.969206 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:42.969171 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/25f26c6e-84cf-4e69-a8e6-97800ebc437f-metrics-cert\") pod \"lws-controller-manager-cbcf9f5f9-rvbxp\" (UID: \"25f26c6e-84cf-4e69-a8e6-97800ebc437f\") " pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:42.969393 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:42.969217 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/25f26c6e-84cf-4e69-a8e6-97800ebc437f-manager-config\") pod \"lws-controller-manager-cbcf9f5f9-rvbxp\" (UID: \"25f26c6e-84cf-4e69-a8e6-97800ebc437f\") " pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:42.969393 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:42.969249 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25f26c6e-84cf-4e69-a8e6-97800ebc437f-cert\") pod \"lws-controller-manager-cbcf9f5f9-rvbxp\" (UID: \"25f26c6e-84cf-4e69-a8e6-97800ebc437f\") " pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:42.969393 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:42.969290 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx7gf\" (UniqueName: \"kubernetes.io/projected/25f26c6e-84cf-4e69-a8e6-97800ebc437f-kube-api-access-fx7gf\") pod \"lws-controller-manager-cbcf9f5f9-rvbxp\" (UID: \"25f26c6e-84cf-4e69-a8e6-97800ebc437f\") " pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:43.015758 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:43.015724 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp"] Jul 28 19:42:43.070044 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:43.070012 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/25f26c6e-84cf-4e69-a8e6-97800ebc437f-metrics-cert\") pod \"lws-controller-manager-cbcf9f5f9-rvbxp\" (UID: \"25f26c6e-84cf-4e69-a8e6-97800ebc437f\") " pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:43.070203 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:43.070055 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/25f26c6e-84cf-4e69-a8e6-97800ebc437f-manager-config\") pod \"lws-controller-manager-cbcf9f5f9-rvbxp\" (UID: \"25f26c6e-84cf-4e69-a8e6-97800ebc437f\") " pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:43.070203 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:43.070082 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25f26c6e-84cf-4e69-a8e6-97800ebc437f-cert\") pod \"lws-controller-manager-cbcf9f5f9-rvbxp\" (UID: \"25f26c6e-84cf-4e69-a8e6-97800ebc437f\") " pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:43.070203 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:43.070126 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx7gf\" (UniqueName: \"kubernetes.io/projected/25f26c6e-84cf-4e69-a8e6-97800ebc437f-kube-api-access-fx7gf\") pod \"lws-controller-manager-cbcf9f5f9-rvbxp\" (UID: \"25f26c6e-84cf-4e69-a8e6-97800ebc437f\") " pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:43.070786 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:43.070769 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/25f26c6e-84cf-4e69-a8e6-97800ebc437f-manager-config\") pod \"lws-controller-manager-cbcf9f5f9-rvbxp\" (UID: \"25f26c6e-84cf-4e69-a8e6-97800ebc437f\") " pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:43.072615 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:43.072591 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25f26c6e-84cf-4e69-a8e6-97800ebc437f-cert\") pod \"lws-controller-manager-cbcf9f5f9-rvbxp\" (UID: \"25f26c6e-84cf-4e69-a8e6-97800ebc437f\") " pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:43.072727 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:43.072656 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/25f26c6e-84cf-4e69-a8e6-97800ebc437f-metrics-cert\") pod \"lws-controller-manager-cbcf9f5f9-rvbxp\" (UID: \"25f26c6e-84cf-4e69-a8e6-97800ebc437f\") " pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:43.102006 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:43.101975 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx7gf\" (UniqueName: \"kubernetes.io/projected/25f26c6e-84cf-4e69-a8e6-97800ebc437f-kube-api-access-fx7gf\") pod \"lws-controller-manager-cbcf9f5f9-rvbxp\" (UID: \"25f26c6e-84cf-4e69-a8e6-97800ebc437f\") " pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:43.179744 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:43.179654 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:43.319081 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:43.319054 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp"] Jul 28 19:42:43.321024 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:42:43.321001 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25f26c6e_84cf_4e69_a8e6_97800ebc437f.slice/crio-47e6d116c4bdad563dac03a0cd208afb1c76479f9ba8a775e545f88d27b62b28 WatchSource:0}: Error finding container 47e6d116c4bdad563dac03a0cd208afb1c76479f9ba8a775e545f88d27b62b28: Status 404 returned error can't find the container with id 47e6d116c4bdad563dac03a0cd208afb1c76479f9ba8a775e545f88d27b62b28 Jul 28 19:42:43.651279 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:43.651242 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" event={"ID":"25f26c6e-84cf-4e69-a8e6-97800ebc437f","Type":"ContainerStarted","Data":"47e6d116c4bdad563dac03a0cd208afb1c76479f9ba8a775e545f88d27b62b28"} Jul 28 19:42:45.662085 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:45.662038 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" event={"ID":"25f26c6e-84cf-4e69-a8e6-97800ebc437f","Type":"ContainerStarted","Data":"05bde1280e0ec66bca4fe9956e3f998894a751a85ac8da004b1f408ddafdeb4c"} Jul 28 19:42:45.662516 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:45.662174 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:45.680185 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:45.680141 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" podStartSLOduration=1.851430919 podStartE2EDuration="3.680125966s" totalImagesPullingTime="1.828695047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:45.679122171 +0000 UTC m=+361.386334064" watchObservedRunningTime="2026-07-28 19:42:45.680125966 +0000 UTC m=+361.387337860" Jul 28 19:42:50.187536 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.187502 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp"] Jul 28 19:42:50.191031 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.191012 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.194613 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.194589 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 28 19:42:50.194757 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.194709 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-szds5\"" Jul 28 19:42:50.195258 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.195232 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 28 19:42:50.195347 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.195274 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 28 19:42:50.219095 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.219059 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp"] Jul 28 19:42:50.333418 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.333386 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c60d1426-7a09-40e7-b11d-7db3f87b0b05-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.333613 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.333427 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8c94\" (UniqueName: \"kubernetes.io/projected/c60d1426-7a09-40e7-b11d-7db3f87b0b05-kube-api-access-n8c94\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.333613 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.333465 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60d1426-7a09-40e7-b11d-7db3f87b0b05-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.333613 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.333544 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c60d1426-7a09-40e7-b11d-7db3f87b0b05-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.333818 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.333665 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60d1426-7a09-40e7-b11d-7db3f87b0b05-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.333818 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.333697 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c60d1426-7a09-40e7-b11d-7db3f87b0b05-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.333818 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.333779 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c60d1426-7a09-40e7-b11d-7db3f87b0b05-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.333934 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.333823 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c60d1426-7a09-40e7-b11d-7db3f87b0b05-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.333934 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.333844 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c60d1426-7a09-40e7-b11d-7db3f87b0b05-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.434405 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.434369 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60d1426-7a09-40e7-b11d-7db3f87b0b05-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.434405 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.434405 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c60d1426-7a09-40e7-b11d-7db3f87b0b05-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.434669 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.434438 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c60d1426-7a09-40e7-b11d-7db3f87b0b05-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.434669 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.434468 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c60d1426-7a09-40e7-b11d-7db3f87b0b05-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.434669 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.434490 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c60d1426-7a09-40e7-b11d-7db3f87b0b05-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.434669 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.434512 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c60d1426-7a09-40e7-b11d-7db3f87b0b05-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.434669 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.434582 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8c94\" (UniqueName: \"kubernetes.io/projected/c60d1426-7a09-40e7-b11d-7db3f87b0b05-kube-api-access-n8c94\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.434669 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.434618 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60d1426-7a09-40e7-b11d-7db3f87b0b05-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.434669 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.434663 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c60d1426-7a09-40e7-b11d-7db3f87b0b05-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.435072 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.434862 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60d1426-7a09-40e7-b11d-7db3f87b0b05-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.435072 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.434888 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c60d1426-7a09-40e7-b11d-7db3f87b0b05-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.435072 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.434970 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c60d1426-7a09-40e7-b11d-7db3f87b0b05-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.435235 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.435124 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60d1426-7a09-40e7-b11d-7db3f87b0b05-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.435297 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.435276 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c60d1426-7a09-40e7-b11d-7db3f87b0b05-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.436989 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.436967 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c60d1426-7a09-40e7-b11d-7db3f87b0b05-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.437184 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.437166 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c60d1426-7a09-40e7-b11d-7db3f87b0b05-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.450318 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.450261 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c60d1426-7a09-40e7-b11d-7db3f87b0b05-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.450471 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.450450 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8c94\" (UniqueName: \"kubernetes.io/projected/c60d1426-7a09-40e7-b11d-7db3f87b0b05-kube-api-access-n8c94\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp\" (UID: \"c60d1426-7a09-40e7-b11d-7db3f87b0b05\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.501407 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.501377 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:50.650820 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.650801 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp"] Jul 28 19:42:50.653089 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:42:50.653063 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc60d1426_7a09_40e7_b11d_7db3f87b0b05.slice/crio-8d6ab0e64bf01daa2a2c194cda3418c42b8929387e5e0c08339f842cd8a4b9f7 WatchSource:0}: Error finding container 8d6ab0e64bf01daa2a2c194cda3418c42b8929387e5e0c08339f842cd8a4b9f7: Status 404 returned error can't find the container with id 8d6ab0e64bf01daa2a2c194cda3418c42b8929387e5e0c08339f842cd8a4b9f7 Jul 28 19:42:50.684373 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.684342 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" event={"ID":"c60d1426-7a09-40e7-b11d-7db3f87b0b05","Type":"ContainerStarted","Data":"8d6ab0e64bf01daa2a2c194cda3418c42b8929387e5e0c08339f842cd8a4b9f7"} Jul 28 19:42:50.869154 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.869121 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq"] Jul 28 19:42:50.873951 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.873926 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:50.882451 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.881840 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 28 19:42:50.882451 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.882086 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 28 19:42:50.882451 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.882279 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bmgxf\"" Jul 28 19:42:50.886421 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.885889 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 28 19:42:50.897260 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:50.897219 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq"] Jul 28 19:42:51.041046 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.040848 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ca3d02d-68c6-44da-940e-1ccb46c31117-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.041046 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.040925 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ca3d02d-68c6-44da-940e-1ccb46c31117-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.041343 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.041196 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ca3d02d-68c6-44da-940e-1ccb46c31117-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.041343 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.041246 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1ca3d02d-68c6-44da-940e-1ccb46c31117-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.041343 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.041291 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1ca3d02d-68c6-44da-940e-1ccb46c31117-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.041343 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.041323 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1ca3d02d-68c6-44da-940e-1ccb46c31117-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.041541 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.041353 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjmvt\" (UniqueName: \"kubernetes.io/projected/1ca3d02d-68c6-44da-940e-1ccb46c31117-kube-api-access-mjmvt\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.142298 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.142205 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ca3d02d-68c6-44da-940e-1ccb46c31117-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.142298 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.142246 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ca3d02d-68c6-44da-940e-1ccb46c31117-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.142298 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.142267 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ca3d02d-68c6-44da-940e-1ccb46c31117-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.142298 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.142288 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1ca3d02d-68c6-44da-940e-1ccb46c31117-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.142662 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.142325 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1ca3d02d-68c6-44da-940e-1ccb46c31117-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.142662 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.142346 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1ca3d02d-68c6-44da-940e-1ccb46c31117-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.142662 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.142368 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjmvt\" (UniqueName: \"kubernetes.io/projected/1ca3d02d-68c6-44da-940e-1ccb46c31117-kube-api-access-mjmvt\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.143247 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.143219 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1ca3d02d-68c6-44da-940e-1ccb46c31117-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.145287 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.145262 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1ca3d02d-68c6-44da-940e-1ccb46c31117-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.145379 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.145302 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1ca3d02d-68c6-44da-940e-1ccb46c31117-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.145379 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.145309 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ca3d02d-68c6-44da-940e-1ccb46c31117-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.145451 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.145404 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ca3d02d-68c6-44da-940e-1ccb46c31117-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.176065 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.176033 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ca3d02d-68c6-44da-940e-1ccb46c31117-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.176197 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.176176 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjmvt\" (UniqueName: \"kubernetes.io/projected/1ca3d02d-68c6-44da-940e-1ccb46c31117-kube-api-access-mjmvt\") pod \"istiod-openshift-gateway-5677ccb49b-ghhnq\" (UID: \"1ca3d02d-68c6-44da-940e-1ccb46c31117\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.189048 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.189020 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:51.333909 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.333875 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq"] Jul 28 19:42:51.337051 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:42:51.337017 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ca3d02d_68c6_44da_940e_1ccb46c31117.slice/crio-ce5ab6adf5cd2cee879baa6dbae6bc4974dc16c1b7078a50aaeff738653f09a5 WatchSource:0}: Error finding container ce5ab6adf5cd2cee879baa6dbae6bc4974dc16c1b7078a50aaeff738653f09a5: Status 404 returned error can't find the container with id ce5ab6adf5cd2cee879baa6dbae6bc4974dc16c1b7078a50aaeff738653f09a5 Jul 28 19:42:51.693167 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:51.693122 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" event={"ID":"1ca3d02d-68c6-44da-940e-1ccb46c31117","Type":"ContainerStarted","Data":"ce5ab6adf5cd2cee879baa6dbae6bc4974dc16c1b7078a50aaeff738653f09a5"} Jul 28 19:42:54.653005 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:54.652966 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 19:42:54.653365 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:54.653058 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 19:42:54.653365 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:54.653098 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 19:42:54.654204 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:54.654180 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 19:42:54.654294 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:54.654236 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 19:42:55.715830 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:55.715797 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" event={"ID":"1ca3d02d-68c6-44da-940e-1ccb46c31117","Type":"ContainerStarted","Data":"1dfd2a69b8cec3bf9a6847eb20bc4bb0802a7767042315ffc9e9d537b0d0b6d3"} Jul 28 19:42:55.716330 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:55.715953 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:55.717471 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:55.717444 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" event={"ID":"c60d1426-7a09-40e7-b11d-7db3f87b0b05","Type":"ContainerStarted","Data":"a1a9cddd06e847edc8f99409681698b530cbaf3cf0af49d99bb8bee0a5732397"} Jul 28 19:42:55.717820 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:55.717806 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" Jul 28 19:42:55.746296 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:55.746248 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ghhnq" podStartSLOduration=2.431724797 podStartE2EDuration="5.74623233s" totalImagesPullingTime="3.314507533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:55.743501438 +0000 UTC m=+371.450713331" watchObservedRunningTime="2026-07-28 19:42:55.74623233 +0000 UTC m=+371.453444223" Jul 28 19:42:55.789874 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:55.789822 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" podStartSLOduration=1.792164427 podStartE2EDuration="5.789802397s" totalImagesPullingTime="3.99763797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:55.789390682 +0000 UTC m=+371.496602574" watchObservedRunningTime="2026-07-28 19:42:55.789802397 +0000 UTC m=+371.497014291" Jul 28 19:42:56.501717 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:56.501673 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:56.506393 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:56.506369 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:56.668623 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:56.668595 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-cbcf9f5f9-rvbxp" Jul 28 19:42:56.722358 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:56.722330 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:42:56.723303 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:42:56.723286 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp" Jul 28 19:44:22.209440 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.209405 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fd674549c-qc52g"] Jul 28 19:44:22.213020 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.212999 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.223873 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.223849 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fd674549c-qc52g"] Jul 28 19:44:22.333621 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.333575 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35fab412-7af6-45e9-a246-00946d6efe8f-service-ca\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.333621 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.333621 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35fab412-7af6-45e9-a246-00946d6efe8f-oauth-serving-cert\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.333910 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.333675 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35fab412-7af6-45e9-a246-00946d6efe8f-console-config\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.333910 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.333699 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzrc6\" (UniqueName: \"kubernetes.io/projected/35fab412-7af6-45e9-a246-00946d6efe8f-kube-api-access-nzrc6\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.333910 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.333735 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35fab412-7af6-45e9-a246-00946d6efe8f-console-oauth-config\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.333910 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.333793 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35fab412-7af6-45e9-a246-00946d6efe8f-trusted-ca-bundle\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.333910 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.333875 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35fab412-7af6-45e9-a246-00946d6efe8f-console-serving-cert\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.435012 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.434980 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35fab412-7af6-45e9-a246-00946d6efe8f-service-ca\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.435012 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.435016 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35fab412-7af6-45e9-a246-00946d6efe8f-oauth-serving-cert\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.435265 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.435038 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35fab412-7af6-45e9-a246-00946d6efe8f-console-config\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.435265 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.435066 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzrc6\" (UniqueName: \"kubernetes.io/projected/35fab412-7af6-45e9-a246-00946d6efe8f-kube-api-access-nzrc6\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.435265 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.435094 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35fab412-7af6-45e9-a246-00946d6efe8f-console-oauth-config\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.435265 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.435120 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35fab412-7af6-45e9-a246-00946d6efe8f-trusted-ca-bundle\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.435265 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.435221 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35fab412-7af6-45e9-a246-00946d6efe8f-console-serving-cert\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.435894 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.435858 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35fab412-7af6-45e9-a246-00946d6efe8f-console-config\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.435894 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.435877 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35fab412-7af6-45e9-a246-00946d6efe8f-oauth-serving-cert\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.436042 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.435946 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35fab412-7af6-45e9-a246-00946d6efe8f-service-ca\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.436133 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.436108 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35fab412-7af6-45e9-a246-00946d6efe8f-trusted-ca-bundle\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.437848 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.437815 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35fab412-7af6-45e9-a246-00946d6efe8f-console-oauth-config\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.437961 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.437821 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35fab412-7af6-45e9-a246-00946d6efe8f-console-serving-cert\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.446919 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.446900 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzrc6\" (UniqueName: \"kubernetes.io/projected/35fab412-7af6-45e9-a246-00946d6efe8f-kube-api-access-nzrc6\") pod \"console-5fd674549c-qc52g\" (UID: \"35fab412-7af6-45e9-a246-00946d6efe8f\") " pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.522753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.522676 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:22.661487 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:22.661465 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fd674549c-qc52g"] Jul 28 19:44:22.664134 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:44:22.664106 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35fab412_7af6_45e9_a246_00946d6efe8f.slice/crio-4f1bd10dd925294d6001238394f35b96fae5b88e3559a4f226b1d398b23a6e87 WatchSource:0}: Error finding container 4f1bd10dd925294d6001238394f35b96fae5b88e3559a4f226b1d398b23a6e87: Status 404 returned error can't find the container with id 4f1bd10dd925294d6001238394f35b96fae5b88e3559a4f226b1d398b23a6e87 Jul 28 19:44:23.121718 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:23.121683 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fd674549c-qc52g" event={"ID":"35fab412-7af6-45e9-a246-00946d6efe8f","Type":"ContainerStarted","Data":"d24f5bf087b31194c32ffb735647b60f726a4c335c4b9d2ed7673114f3d38669"} Jul 28 19:44:23.121919 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:23.121733 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fd674549c-qc52g" event={"ID":"35fab412-7af6-45e9-a246-00946d6efe8f","Type":"ContainerStarted","Data":"4f1bd10dd925294d6001238394f35b96fae5b88e3559a4f226b1d398b23a6e87"} Jul 28 19:44:23.143706 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:23.143658 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fd674549c-qc52g" podStartSLOduration=1.143624025 podStartE2EDuration="1.143624025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:23.142064305 +0000 UTC m=+458.849276198" watchObservedRunningTime="2026-07-28 19:44:23.143624025 +0000 UTC m=+458.850835917" Jul 28 19:44:32.523299 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:32.523264 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:32.523700 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:32.523334 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:32.527933 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:32.527910 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:33.172909 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:33.172880 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5fd674549c-qc52g" Jul 28 19:44:33.226292 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:33.226258 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ff495695f-gq5md"] Jul 28 19:44:44.606080 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.606047 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs"] Jul 28 19:44:44.609564 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.609548 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs" Jul 28 19:44:44.614378 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.613184 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 28 19:44:44.614378 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.613966 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 28 19:44:44.614378 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.614259 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8qgj2\"" Jul 28 19:44:44.629759 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.629737 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btsld\" (UniqueName: \"kubernetes.io/projected/a8955edf-0fdc-437f-b687-3aa4376a9294-kube-api-access-btsld\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-78pxs\" (UID: \"a8955edf-0fdc-437f-b687-3aa4376a9294\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs" Jul 28 19:44:44.629886 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.629802 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8955edf-0fdc-437f-b687-3aa4376a9294-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-78pxs\" (UID: \"a8955edf-0fdc-437f-b687-3aa4376a9294\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs" Jul 28 19:44:44.633838 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.633815 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs"] Jul 28 19:44:44.730309 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.730282 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btsld\" (UniqueName: \"kubernetes.io/projected/a8955edf-0fdc-437f-b687-3aa4376a9294-kube-api-access-btsld\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-78pxs\" (UID: \"a8955edf-0fdc-437f-b687-3aa4376a9294\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs" Jul 28 19:44:44.730464 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.730329 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8955edf-0fdc-437f-b687-3aa4376a9294-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-78pxs\" (UID: \"a8955edf-0fdc-437f-b687-3aa4376a9294\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs" Jul 28 19:44:44.730697 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.730681 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8955edf-0fdc-437f-b687-3aa4376a9294-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-78pxs\" (UID: \"a8955edf-0fdc-437f-b687-3aa4376a9294\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs" Jul 28 19:44:44.740764 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.740743 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 28 19:44:44.751838 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.751815 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 28 19:44:44.753550 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.753533 2564 scope.go:117] "RemoveContainer" containerID="287bf55a2294e31875c6b7c6040290e141975e9f271fa3b1f80a8142404fc3c9" Jul 28 19:44:44.761947 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.761921 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btsld\" (UniqueName: \"kubernetes.io/projected/a8955edf-0fdc-437f-b687-3aa4376a9294-kube-api-access-btsld\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-78pxs\" (UID: \"a8955edf-0fdc-437f-b687-3aa4376a9294\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs" Jul 28 19:44:44.926505 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.926426 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8qgj2\"" Jul 28 19:44:44.934030 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:44.934010 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs" Jul 28 19:44:45.081159 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:45.081137 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs"] Jul 28 19:44:45.083598 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:44:45.083565 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8955edf_0fdc_437f_b687_3aa4376a9294.slice/crio-02431aec166ecc76de5624d4ed829d8c096b45867c912617901040f767e24475 WatchSource:0}: Error finding container 02431aec166ecc76de5624d4ed829d8c096b45867c912617901040f767e24475: Status 404 returned error can't find the container with id 02431aec166ecc76de5624d4ed829d8c096b45867c912617901040f767e24475 Jul 28 19:44:45.227423 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:45.227327 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs" event={"ID":"a8955edf-0fdc-437f-b687-3aa4376a9294","Type":"ContainerStarted","Data":"02431aec166ecc76de5624d4ed829d8c096b45867c912617901040f767e24475"} Jul 28 19:44:45.411524 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:45.411486 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs"] Jul 28 19:44:45.416711 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:45.416687 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs"] Jul 28 19:44:45.506061 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:45.505985 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xx8mx"] Jul 28 19:44:45.509853 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:45.509831 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xx8mx" Jul 28 19:44:45.515581 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:45.515562 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-dbs6w\"" Jul 28 19:44:45.523625 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:45.523591 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xx8mx"] Jul 28 19:44:45.536785 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:45.536754 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz8kz\" (UniqueName: \"kubernetes.io/projected/0f26d5eb-a761-4b32-b0eb-31374b3c8cdc-kube-api-access-gz8kz\") pod \"limitador-operator-controller-manager-859d7b78bd-xx8mx\" (UID: \"0f26d5eb-a761-4b32-b0eb-31374b3c8cdc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xx8mx" Jul 28 19:44:45.637289 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:45.637256 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz8kz\" (UniqueName: \"kubernetes.io/projected/0f26d5eb-a761-4b32-b0eb-31374b3c8cdc-kube-api-access-gz8kz\") pod \"limitador-operator-controller-manager-859d7b78bd-xx8mx\" (UID: \"0f26d5eb-a761-4b32-b0eb-31374b3c8cdc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xx8mx" Jul 28 19:44:45.647168 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:45.647146 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz8kz\" (UniqueName: \"kubernetes.io/projected/0f26d5eb-a761-4b32-b0eb-31374b3c8cdc-kube-api-access-gz8kz\") pod \"limitador-operator-controller-manager-859d7b78bd-xx8mx\" (UID: \"0f26d5eb-a761-4b32-b0eb-31374b3c8cdc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xx8mx" Jul 28 19:44:45.825445 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:45.825409 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xx8mx" Jul 28 19:44:46.207759 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:46.207710 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xx8mx"] Jul 28 19:44:46.219589 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:44:46.219559 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f26d5eb_a761_4b32_b0eb_31374b3c8cdc.slice/crio-757418a1a83c2329f5c70ca8173997f70ada4a9278daccb00c050fb9be9d59df WatchSource:0}: Error finding container 757418a1a83c2329f5c70ca8173997f70ada4a9278daccb00c050fb9be9d59df: Status 404 returned error can't find the container with id 757418a1a83c2329f5c70ca8173997f70ada4a9278daccb00c050fb9be9d59df Jul 28 19:44:46.232613 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:46.232586 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xx8mx" event={"ID":"0f26d5eb-a761-4b32-b0eb-31374b3c8cdc","Type":"ContainerStarted","Data":"757418a1a83c2329f5c70ca8173997f70ada4a9278daccb00c050fb9be9d59df"} Jul 28 19:44:48.244820 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:48.244785 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xx8mx" event={"ID":"0f26d5eb-a761-4b32-b0eb-31374b3c8cdc","Type":"ContainerStarted","Data":"2e49b5458276e1f01c988edd626587e0213c8506762570a293f6f6a1ff34b7b9"} Jul 28 19:44:48.245210 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:48.244899 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xx8mx" Jul 28 19:44:48.270420 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:48.270371 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xx8mx" podStartSLOduration=1.7893273170000001 podStartE2EDuration="3.270357608s" totalImagesPullingTime="1.481030291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:48.268970078 +0000 UTC m=+483.976181971" watchObservedRunningTime="2026-07-28 19:44:48.270357608 +0000 UTC m=+483.977569500" Jul 28 19:44:50.257170 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:50.257051 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs" podUID="a8955edf-0fdc-437f-b687-3aa4376a9294" containerName="manager" containerID="cri-o://4fde0020fb76a8d66a0d4a6911b48c0bcb31c4cbf5acf104cd88ac2e9c57376b" gracePeriod=2 Jul 28 19:44:50.491409 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:50.491388 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs" Jul 28 19:44:50.494048 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:44:50.494023 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-78pxs\" is forbidden: User \"system:node:ip-10-0-136-194.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-194.ec2.internal' and this object" podUID="a8955edf-0fdc-437f-b687-3aa4376a9294" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs" Jul 28 19:44:50.575139 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:50.575111 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btsld\" (UniqueName: \"kubernetes.io/projected/a8955edf-0fdc-437f-b687-3aa4376a9294-kube-api-access-btsld\") pod \"a8955edf-0fdc-437f-b687-3aa4376a9294\" (UID: \"a8955edf-0fdc-437f-b687-3aa4376a9294\") " Jul 28 19:44:50.575273 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:50.575143 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8955edf-0fdc-437f-b687-3aa4376a9294-extensions-socket-volume\") pod \"a8955edf-0fdc-437f-b687-3aa4376a9294\" (UID: \"a8955edf-0fdc-437f-b687-3aa4376a9294\") " Jul 28 19:44:50.575448 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:50.575420 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8955edf-0fdc-437f-b687-3aa4376a9294-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a8955edf-0fdc-437f-b687-3aa4376a9294" (UID: "a8955edf-0fdc-437f-b687-3aa4376a9294"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 19:44:50.577179 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:50.577157 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8955edf-0fdc-437f-b687-3aa4376a9294-kube-api-access-btsld" (OuterVolumeSpecName: "kube-api-access-btsld") pod "a8955edf-0fdc-437f-b687-3aa4376a9294" (UID: "a8955edf-0fdc-437f-b687-3aa4376a9294"). InnerVolumeSpecName "kube-api-access-btsld". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 19:44:50.675970 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:50.675930 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btsld\" (UniqueName: \"kubernetes.io/projected/a8955edf-0fdc-437f-b687-3aa4376a9294-kube-api-access-btsld\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:44:50.675970 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:50.675964 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8955edf-0fdc-437f-b687-3aa4376a9294-extensions-socket-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:44:50.918540 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:50.918454 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8955edf-0fdc-437f-b687-3aa4376a9294" path="/var/lib/kubelet/pods/a8955edf-0fdc-437f-b687-3aa4376a9294/volumes" Jul 28 19:44:51.263469 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:51.263384 2564 generic.go:356] "Generic (PLEG): container finished" podID="a8955edf-0fdc-437f-b687-3aa4376a9294" containerID="4fde0020fb76a8d66a0d4a6911b48c0bcb31c4cbf5acf104cd88ac2e9c57376b" exitCode=2 Jul 28 19:44:51.263469 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:51.263429 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs" Jul 28 19:44:51.263926 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:51.263485 2564 scope.go:117] "RemoveContainer" containerID="4fde0020fb76a8d66a0d4a6911b48c0bcb31c4cbf5acf104cd88ac2e9c57376b" Jul 28 19:44:51.268273 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:44:51.268247 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-78pxs\" is forbidden: User \"system:node:ip-10-0-136-194.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-194.ec2.internal' and this object" podUID="a8955edf-0fdc-437f-b687-3aa4376a9294" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-78pxs" Jul 28 19:44:51.272204 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:51.272192 2564 scope.go:117] "RemoveContainer" containerID="4fde0020fb76a8d66a0d4a6911b48c0bcb31c4cbf5acf104cd88ac2e9c57376b" Jul 28 19:44:51.272448 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:44:51.272431 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fde0020fb76a8d66a0d4a6911b48c0bcb31c4cbf5acf104cd88ac2e9c57376b\": container with ID starting with 4fde0020fb76a8d66a0d4a6911b48c0bcb31c4cbf5acf104cd88ac2e9c57376b not found: ID does not exist" containerID="4fde0020fb76a8d66a0d4a6911b48c0bcb31c4cbf5acf104cd88ac2e9c57376b" Jul 28 19:44:51.272498 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:51.272456 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fde0020fb76a8d66a0d4a6911b48c0bcb31c4cbf5acf104cd88ac2e9c57376b"} err="failed to get container status \"4fde0020fb76a8d66a0d4a6911b48c0bcb31c4cbf5acf104cd88ac2e9c57376b\": rpc error: code = NotFound desc = could not find container \"4fde0020fb76a8d66a0d4a6911b48c0bcb31c4cbf5acf104cd88ac2e9c57376b\": container with ID starting with 4fde0020fb76a8d66a0d4a6911b48c0bcb31c4cbf5acf104cd88ac2e9c57376b not found: ID does not exist" Jul 28 19:44:58.246384 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.246337 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7ff495695f-gq5md" podUID="1df4aefd-3e9b-4065-aa4e-38165545bdab" containerName="console" containerID="cri-o://21c8b9c659960ef1c4f03933e79730cbcbdb1892ca6c97c2d2c10740f8751d71" gracePeriod=15 Jul 28 19:44:58.491945 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.491920 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:44:58.549191 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.549162 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dlg6\" (UniqueName: \"kubernetes.io/projected/1df4aefd-3e9b-4065-aa4e-38165545bdab-kube-api-access-6dlg6\") pod \"1df4aefd-3e9b-4065-aa4e-38165545bdab\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " Jul 28 19:44:58.549358 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.549233 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-trusted-ca-bundle\") pod \"1df4aefd-3e9b-4065-aa4e-38165545bdab\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " Jul 28 19:44:58.549427 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.549402 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-oauth-config\") pod \"1df4aefd-3e9b-4065-aa4e-38165545bdab\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " Jul 28 19:44:58.549475 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.549457 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-service-ca\") pod \"1df4aefd-3e9b-4065-aa4e-38165545bdab\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " Jul 28 19:44:58.549528 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.549488 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-config\") pod \"1df4aefd-3e9b-4065-aa4e-38165545bdab\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " Jul 28 19:44:58.549528 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.549515 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-oauth-serving-cert\") pod \"1df4aefd-3e9b-4065-aa4e-38165545bdab\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " Jul 28 19:44:58.549659 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.549555 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-serving-cert\") pod \"1df4aefd-3e9b-4065-aa4e-38165545bdab\" (UID: \"1df4aefd-3e9b-4065-aa4e-38165545bdab\") " Jul 28 19:44:58.549659 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.549573 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1df4aefd-3e9b-4065-aa4e-38165545bdab" (UID: "1df4aefd-3e9b-4065-aa4e-38165545bdab"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:44:58.550228 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.550132 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-service-ca" (OuterVolumeSpecName: "service-ca") pod "1df4aefd-3e9b-4065-aa4e-38165545bdab" (UID: "1df4aefd-3e9b-4065-aa4e-38165545bdab"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:44:58.550228 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.550168 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-config" (OuterVolumeSpecName: "console-config") pod "1df4aefd-3e9b-4065-aa4e-38165545bdab" (UID: "1df4aefd-3e9b-4065-aa4e-38165545bdab"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:44:58.550470 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.550324 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-trusted-ca-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:44:58.550470 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.550354 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:44:58.550470 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.550367 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:44:58.550470 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.550419 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1df4aefd-3e9b-4065-aa4e-38165545bdab" (UID: "1df4aefd-3e9b-4065-aa4e-38165545bdab"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 19:44:58.552181 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.551857 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1df4aefd-3e9b-4065-aa4e-38165545bdab-kube-api-access-6dlg6" (OuterVolumeSpecName: "kube-api-access-6dlg6") pod "1df4aefd-3e9b-4065-aa4e-38165545bdab" (UID: "1df4aefd-3e9b-4065-aa4e-38165545bdab"). InnerVolumeSpecName "kube-api-access-6dlg6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 19:44:58.557917 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.557895 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1df4aefd-3e9b-4065-aa4e-38165545bdab" (UID: "1df4aefd-3e9b-4065-aa4e-38165545bdab"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:44:58.558164 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.558151 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1df4aefd-3e9b-4065-aa4e-38165545bdab" (UID: "1df4aefd-3e9b-4065-aa4e-38165545bdab"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 19:44:58.651120 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.651084 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dlg6\" (UniqueName: \"kubernetes.io/projected/1df4aefd-3e9b-4065-aa4e-38165545bdab-kube-api-access-6dlg6\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:44:58.651120 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.651115 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-oauth-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:44:58.651120 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.651125 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1df4aefd-3e9b-4065-aa4e-38165545bdab-oauth-serving-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:44:58.651345 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:58.651133 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1df4aefd-3e9b-4065-aa4e-38165545bdab-console-serving-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 19:44:59.253135 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:59.253061 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xx8mx" Jul 28 19:44:59.302659 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:59.302611 2564 generic.go:356] "Generic (PLEG): container finished" podID="1df4aefd-3e9b-4065-aa4e-38165545bdab" containerID="21c8b9c659960ef1c4f03933e79730cbcbdb1892ca6c97c2d2c10740f8751d71" exitCode=0 Jul 28 19:44:59.302829 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:59.302668 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff495695f-gq5md" event={"ID":"1df4aefd-3e9b-4065-aa4e-38165545bdab","Type":"ContainerDied","Data":"21c8b9c659960ef1c4f03933e79730cbcbdb1892ca6c97c2d2c10740f8751d71"} Jul 28 19:44:59.302829 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:59.302730 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff495695f-gq5md" event={"ID":"1df4aefd-3e9b-4065-aa4e-38165545bdab","Type":"ContainerDied","Data":"04596b0f74a9421102373f77bcf77ab2fcc118b88223aa55a9b9cbc8cb8cbb10"} Jul 28 19:44:59.302829 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:59.302750 2564 scope.go:117] "RemoveContainer" containerID="21c8b9c659960ef1c4f03933e79730cbcbdb1892ca6c97c2d2c10740f8751d71" Jul 28 19:44:59.302829 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:59.302688 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff495695f-gq5md" Jul 28 19:44:59.311366 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:59.311344 2564 scope.go:117] "RemoveContainer" containerID="21c8b9c659960ef1c4f03933e79730cbcbdb1892ca6c97c2d2c10740f8751d71" Jul 28 19:44:59.311630 ip-10-0-136-194 kubenswrapper[2564]: E0728 19:44:59.311611 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21c8b9c659960ef1c4f03933e79730cbcbdb1892ca6c97c2d2c10740f8751d71\": container with ID starting with 21c8b9c659960ef1c4f03933e79730cbcbdb1892ca6c97c2d2c10740f8751d71 not found: ID does not exist" containerID="21c8b9c659960ef1c4f03933e79730cbcbdb1892ca6c97c2d2c10740f8751d71" Jul 28 19:44:59.311738 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:59.311656 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21c8b9c659960ef1c4f03933e79730cbcbdb1892ca6c97c2d2c10740f8751d71"} err="failed to get container status \"21c8b9c659960ef1c4f03933e79730cbcbdb1892ca6c97c2d2c10740f8751d71\": rpc error: code = NotFound desc = could not find container \"21c8b9c659960ef1c4f03933e79730cbcbdb1892ca6c97c2d2c10740f8751d71\": container with ID starting with 21c8b9c659960ef1c4f03933e79730cbcbdb1892ca6c97c2d2c10740f8751d71 not found: ID does not exist" Jul 28 19:44:59.325481 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:59.325456 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ff495695f-gq5md"] Jul 28 19:44:59.331734 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:44:59.331713 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7ff495695f-gq5md"] Jul 28 19:45:00.918109 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:45:00.918075 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1df4aefd-3e9b-4065-aa4e-38165545bdab" path="/var/lib/kubelet/pods/1df4aefd-3e9b-4065-aa4e-38165545bdab/volumes" Jul 28 19:47:16.802417 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.802383 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4"] Jul 28 19:47:16.804867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.802773 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8955edf-0fdc-437f-b687-3aa4376a9294" containerName="manager" Jul 28 19:47:16.804867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.802785 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8955edf-0fdc-437f-b687-3aa4376a9294" containerName="manager" Jul 28 19:47:16.804867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.802802 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1df4aefd-3e9b-4065-aa4e-38165545bdab" containerName="console" Jul 28 19:47:16.804867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.802807 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1df4aefd-3e9b-4065-aa4e-38165545bdab" containerName="console" Jul 28 19:47:16.804867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.802853 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1df4aefd-3e9b-4065-aa4e-38165545bdab" containerName="console" Jul 28 19:47:16.804867 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.802865 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8955edf-0fdc-437f-b687-3aa4376a9294" containerName="manager" Jul 28 19:47:16.805753 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.805728 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:16.809839 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.809811 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 28 19:47:16.809839 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.809809 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 28 19:47:16.810038 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.809848 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 28 19:47:16.810038 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.809858 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5hlzv\"" Jul 28 19:47:16.815595 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.815572 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4"] Jul 28 19:47:16.905047 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.905011 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24e40f82-2efe-4646-b94a-5292a02636cf-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:16.905232 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.905065 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24e40f82-2efe-4646-b94a-5292a02636cf-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:16.905232 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.905118 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmj4m\" (UniqueName: \"kubernetes.io/projected/24e40f82-2efe-4646-b94a-5292a02636cf-kube-api-access-rmj4m\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:16.905232 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.905139 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24e40f82-2efe-4646-b94a-5292a02636cf-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:16.905232 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.905158 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24e40f82-2efe-4646-b94a-5292a02636cf-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:16.905232 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:16.905205 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24e40f82-2efe-4646-b94a-5292a02636cf-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:17.005969 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.005936 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24e40f82-2efe-4646-b94a-5292a02636cf-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:17.005969 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.005967 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmj4m\" (UniqueName: \"kubernetes.io/projected/24e40f82-2efe-4646-b94a-5292a02636cf-kube-api-access-rmj4m\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:17.006192 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.005985 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24e40f82-2efe-4646-b94a-5292a02636cf-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:17.006192 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.006005 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24e40f82-2efe-4646-b94a-5292a02636cf-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:17.006192 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.006129 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24e40f82-2efe-4646-b94a-5292a02636cf-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:17.006308 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.006201 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24e40f82-2efe-4646-b94a-5292a02636cf-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:17.006488 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.006457 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24e40f82-2efe-4646-b94a-5292a02636cf-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:17.006488 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.006484 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24e40f82-2efe-4646-b94a-5292a02636cf-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:17.006726 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.006555 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24e40f82-2efe-4646-b94a-5292a02636cf-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:17.008305 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.008276 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24e40f82-2efe-4646-b94a-5292a02636cf-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:17.008526 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.008504 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24e40f82-2efe-4646-b94a-5292a02636cf-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:17.020268 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.020244 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmj4m\" (UniqueName: \"kubernetes.io/projected/24e40f82-2efe-4646-b94a-5292a02636cf-kube-api-access-rmj4m\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4\" (UID: \"24e40f82-2efe-4646-b94a-5292a02636cf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:17.118700 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.118589 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:17.252616 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.252584 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4"] Jul 28 19:47:17.258964 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.258938 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 19:47:17.933447 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:17.933412 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" event={"ID":"24e40f82-2efe-4646-b94a-5292a02636cf","Type":"ContainerStarted","Data":"1598962d395e7f7fadf4c24ad31ffe8133eae1ee241422f04d8b00c5e0c5ff44"} Jul 28 19:47:18.939797 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:18.939756 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" event={"ID":"24e40f82-2efe-4646-b94a-5292a02636cf","Type":"ContainerStarted","Data":"45c302c4decba04d02c2679fccc8008b81af2255f895f629753395bd8242ebbe"} Jul 28 19:47:18.959761 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:18.959708 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" podStartSLOduration=1.969682057 podStartE2EDuration="2.959692378s" totalImagesPullingTime="990.010321ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:47:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:47:18.95693888 +0000 UTC m=+634.664150777" watchObservedRunningTime="2026-07-28 19:47:18.959692378 +0000 UTC m=+634.666904270" Jul 28 19:47:27.119621 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:27.119577 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:27.120114 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:27.119679 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:27.132380 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:27.132351 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:27.994317 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:27.994287 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4" Jul 28 19:47:46.306306 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.306273 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl"] Jul 28 19:47:46.309755 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.309729 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.312707 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.312684 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 28 19:47:46.318105 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.318084 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl"] Jul 28 19:47:46.381562 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.381534 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.381562 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.381568 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.381781 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.381675 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.381781 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.381720 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.381781 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.381750 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.381781 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.381766 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8rfp\" (UniqueName: \"kubernetes.io/projected/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-kube-api-access-f8rfp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.483116 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.483081 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.483116 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.483118 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.483351 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.483141 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8rfp\" (UniqueName: \"kubernetes.io/projected/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-kube-api-access-f8rfp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.483351 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.483226 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.483351 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.483251 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.483351 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.483309 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.483560 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.483511 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.483620 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.483602 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.483716 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.483685 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.485198 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.485170 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.485570 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.485549 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.491459 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.491440 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8rfp\" (UniqueName: \"kubernetes.io/projected/fd5dafea-e80c-4a87-b4ac-e6e8246ccab8-kube-api-access-f8rfp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwrhl\" (UID: \"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.620860 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.620784 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:46.750514 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:46.750489 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl"] Jul 28 19:47:46.753325 ip-10-0-136-194 kubenswrapper[2564]: W0728 19:47:46.753296 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd5dafea_e80c_4a87_b4ac_e6e8246ccab8.slice/crio-b78d70775c30326ed38ef60b858bf2f50bc77a60fb9ce4556ceee797e92e991b WatchSource:0}: Error finding container b78d70775c30326ed38ef60b858bf2f50bc77a60fb9ce4556ceee797e92e991b: Status 404 returned error can't find the container with id b78d70775c30326ed38ef60b858bf2f50bc77a60fb9ce4556ceee797e92e991b Jul 28 19:47:47.072757 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:47.072724 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" event={"ID":"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8","Type":"ContainerStarted","Data":"b78d70775c30326ed38ef60b858bf2f50bc77a60fb9ce4556ceee797e92e991b"} Jul 28 19:47:48.079554 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:48.079518 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" event={"ID":"fd5dafea-e80c-4a87-b4ac-e6e8246ccab8","Type":"ContainerStarted","Data":"3582c45d38968c33392e8858b636447d325b60e88181cbce0f34da60bcb9a896"} Jul 28 19:47:48.099988 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:48.099941 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" podStartSLOduration=1.8295877 podStartE2EDuration="2.099926662s" totalImagesPullingTime="270.338962ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:47:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:47:48.098608222 +0000 UTC m=+663.805820116" watchObservedRunningTime="2026-07-28 19:47:48.099926662 +0000 UTC m=+663.807138555" Jul 28 19:47:56.621747 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:56.621704 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:56.622155 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:56.621790 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:56.633987 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:56.633966 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 19:47:57.135739 ip-10-0-136-194 kubenswrapper[2564]: I0728 19:47:57.135710 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwrhl" Jul 28 20:05:13.392880 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.392759 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m"] Jul 28 20:05:13.396361 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.396342 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.399708 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.399680 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-81375d1e-openshift-default-dockercfg-nmgmq\"" Jul 28 20:05:13.416395 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.416363 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m"] Jul 28 20:05:13.460848 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.460810 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17692add-12e7-421d-8ef4-a5ccb5e850fe-istiod-ca-cert\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.460848 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.460859 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-envoy\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.461135 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.460894 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-workload-socket\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.461135 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.460995 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-token\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.461135 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.461037 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-workload-certs\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.461135 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.461089 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-podinfo\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.461135 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.461115 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-credential-socket\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.461313 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.461143 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-data\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.461313 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.461164 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrmsn\" (UniqueName: \"kubernetes.io/projected/17692add-12e7-421d-8ef4-a5ccb5e850fe-kube-api-access-jrmsn\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.562710 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.562668 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-podinfo\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.562710 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.562706 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-credential-socket\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.562998 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.562740 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-data\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.562998 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.562766 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrmsn\" (UniqueName: \"kubernetes.io/projected/17692add-12e7-421d-8ef4-a5ccb5e850fe-kube-api-access-jrmsn\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.562998 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.562836 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17692add-12e7-421d-8ef4-a5ccb5e850fe-istiod-ca-cert\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.562998 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.562871 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-envoy\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.562998 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.562919 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-workload-socket\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.562998 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.562964 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-token\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.562998 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.562994 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-workload-certs\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.563371 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.563247 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-data\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.563371 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.563330 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-credential-socket\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.563622 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.563595 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17692add-12e7-421d-8ef4-a5ccb5e850fe-istiod-ca-cert\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.563748 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.563668 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-workload-certs\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.563825 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.563803 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-workload-socket\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.565326 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.565306 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-podinfo\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.565408 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.565375 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-envoy\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.571942 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.571921 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-token\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.572020 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.571993 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrmsn\" (UniqueName: \"kubernetes.io/projected/17692add-12e7-421d-8ef4-a5ccb5e850fe-kube-api-access-jrmsn\") pod \"e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.712993 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.712894 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:13.852188 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.852160 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m"] Jul 28 20:05:13.853975 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:05:13.853941 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17692add_12e7_421d_8ef4_a5ccb5e850fe.slice/crio-4b5b2cbb28951249e2b53c547a4f60819878b580f35359bd13359b92b91a3daf WatchSource:0}: Error finding container 4b5b2cbb28951249e2b53c547a4f60819878b580f35359bd13359b92b91a3daf: Status 404 returned error can't find the container with id 4b5b2cbb28951249e2b53c547a4f60819878b580f35359bd13359b92b91a3daf Jul 28 20:05:13.856237 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.856202 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:05:13.856304 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.856285 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:05:13.856340 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:13.856321 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:05:14.083652 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:14.083606 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" event={"ID":"17692add-12e7-421d-8ef4-a5ccb5e850fe","Type":"ContainerStarted","Data":"7f1c3d0ff75087a24813b08d716fccd5c268396009b5ab3e23da1ed7e8a71a0e"} Jul 28 20:05:14.083871 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:14.083700 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" event={"ID":"17692add-12e7-421d-8ef4-a5ccb5e850fe","Type":"ContainerStarted","Data":"4b5b2cbb28951249e2b53c547a4f60819878b580f35359bd13359b92b91a3daf"} Jul 28 20:05:14.111859 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:14.111808 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" podStartSLOduration=1.111791086 podStartE2EDuration="1.111791086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:05:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:05:14.110456871 +0000 UTC m=+1709.817668766" watchObservedRunningTime="2026-07-28 20:05:14.111791086 +0000 UTC m=+1709.819003051" Jul 28 20:05:14.713949 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:14.713910 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:14.719086 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:14.719061 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:15.088262 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:15.088230 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:15.089102 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:15.089084 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:31.129854 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:31.129799 2564 patch_prober.go:28] interesting pod/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 28 20:05:31.130319 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:31.129870 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" podUID="17692add-12e7-421d-8ef4-a5ccb5e850fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 28 20:05:32.176349 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:32.176315 2564 generic.go:356] "Generic (PLEG): container finished" podID="17692add-12e7-421d-8ef4-a5ccb5e850fe" containerID="7f1c3d0ff75087a24813b08d716fccd5c268396009b5ab3e23da1ed7e8a71a0e" exitCode=0 Jul 28 20:05:32.176770 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:32.176386 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" event={"ID":"17692add-12e7-421d-8ef4-a5ccb5e850fe","Type":"ContainerDied","Data":"7f1c3d0ff75087a24813b08d716fccd5c268396009b5ab3e23da1ed7e8a71a0e"} Jul 28 20:05:32.176817 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:32.176794 2564 scope.go:117] "RemoveContainer" containerID="7f1c3d0ff75087a24813b08d716fccd5c268396009b5ab3e23da1ed7e8a71a0e" Jul 28 20:05:32.177888 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:32.177861 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:05:32.177984 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:32.177926 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:05:32.177984 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:32.177955 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:05:33.183221 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:33.183191 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" event={"ID":"17692add-12e7-421d-8ef4-a5ccb5e850fe","Type":"ContainerStarted","Data":"e24eecee4674868adca25b41bcad6cc15eb505d279bd98a18f7e3e81dcffa872"} Jul 28 20:05:33.713808 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:33.713748 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:33.714007 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:33.713986 2564 patch_prober.go:28] interesting pod/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 28 20:05:33.714054 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:33.714036 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" podUID="17692add-12e7-421d-8ef4-a5ccb5e850fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 28 20:05:34.713849 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:34.713814 2564 patch_prober.go:28] interesting pod/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 28 20:05:34.714318 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:34.713886 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" podUID="17692add-12e7-421d-8ef4-a5ccb5e850fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 28 20:05:35.713432 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:35.713400 2564 patch_prober.go:28] interesting pod/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 28 20:05:35.713598 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:35.713457 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" podUID="17692add-12e7-421d-8ef4-a5ccb5e850fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 28 20:05:36.713474 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:36.713438 2564 patch_prober.go:28] interesting pod/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 28 20:05:36.713925 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:36.713493 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" podUID="17692add-12e7-421d-8ef4-a5ccb5e850fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 28 20:05:37.717315 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:37.717285 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:37.717725 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:37.717618 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:37.718407 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:37.718390 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:46.483095 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:46.483062 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m"] Jul 28 20:05:46.483541 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:46.483271 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" podUID="17692add-12e7-421d-8ef4-a5ccb5e850fe" containerName="istio-proxy" containerID="cri-o://e24eecee4674868adca25b41bcad6cc15eb505d279bd98a18f7e3e81dcffa872" gracePeriod=30 Jul 28 20:05:50.369725 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:50.369689 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z"] Jul 28 20:05:50.372707 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:50.372690 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" Jul 28 20:05:50.375583 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:50.375559 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-bdcbbca7\"" Jul 28 20:05:50.375701 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:50.375561 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-bdcbbca7-dockercfg-5rvfk\"" Jul 28 20:05:50.382868 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:50.382844 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z"] Jul 28 20:05:50.511448 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:50.511413 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fad77ecb-28c4-4bbc-8c5b-cf4596239b0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z\" (UID: \"fad77ecb-28c4-4bbc-8c5b-cf4596239b0b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" Jul 28 20:05:50.511627 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:50.511510 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp4fr\" (UniqueName: \"kubernetes.io/projected/fad77ecb-28c4-4bbc-8c5b-cf4596239b0b-kube-api-access-lp4fr\") pod \"payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z\" (UID: \"fad77ecb-28c4-4bbc-8c5b-cf4596239b0b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" Jul 28 20:05:50.612355 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:50.612313 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fad77ecb-28c4-4bbc-8c5b-cf4596239b0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z\" (UID: \"fad77ecb-28c4-4bbc-8c5b-cf4596239b0b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" Jul 28 20:05:50.612540 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:50.612391 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp4fr\" (UniqueName: \"kubernetes.io/projected/fad77ecb-28c4-4bbc-8c5b-cf4596239b0b-kube-api-access-lp4fr\") pod \"payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z\" (UID: \"fad77ecb-28c4-4bbc-8c5b-cf4596239b0b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" Jul 28 20:05:50.613066 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:50.613046 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fad77ecb-28c4-4bbc-8c5b-cf4596239b0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z\" (UID: \"fad77ecb-28c4-4bbc-8c5b-cf4596239b0b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" Jul 28 20:05:50.620606 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:50.620550 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp4fr\" (UniqueName: \"kubernetes.io/projected/fad77ecb-28c4-4bbc-8c5b-cf4596239b0b-kube-api-access-lp4fr\") pod \"payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z\" (UID: \"fad77ecb-28c4-4bbc-8c5b-cf4596239b0b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" Jul 28 20:05:50.684567 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:50.684528 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" Jul 28 20:05:50.814243 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:50.814088 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z"] Jul 28 20:05:50.817147 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:05:50.817122 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfad77ecb_28c4_4bbc_8c5b_cf4596239b0b.slice/crio-bd1be0dbdad13dce314c992f82bd2c8f55c962bd99ce1b01e5ee13f28de71e69 WatchSource:0}: Error finding container bd1be0dbdad13dce314c992f82bd2c8f55c962bd99ce1b01e5ee13f28de71e69: Status 404 returned error can't find the container with id bd1be0dbdad13dce314c992f82bd2c8f55c962bd99ce1b01e5ee13f28de71e69 Jul 28 20:05:50.819423 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:50.819408 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 20:05:51.270831 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.270797 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" event={"ID":"fad77ecb-28c4-4bbc-8c5b-cf4596239b0b","Type":"ContainerStarted","Data":"bd1be0dbdad13dce314c992f82bd2c8f55c962bd99ce1b01e5ee13f28de71e69"} Jul 28 20:05:51.775327 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.775088 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:51.824098 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.823983 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-workload-socket\") pod \"17692add-12e7-421d-8ef4-a5ccb5e850fe\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " Jul 28 20:05:51.824098 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.824100 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-envoy\") pod \"17692add-12e7-421d-8ef4-a5ccb5e850fe\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " Jul 28 20:05:51.824349 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.824146 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-podinfo\") pod \"17692add-12e7-421d-8ef4-a5ccb5e850fe\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " Jul 28 20:05:51.824349 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.824174 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-token\") pod \"17692add-12e7-421d-8ef4-a5ccb5e850fe\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " Jul 28 20:05:51.824349 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.824209 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-data\") pod \"17692add-12e7-421d-8ef4-a5ccb5e850fe\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " Jul 28 20:05:51.824349 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.824240 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-workload-certs\") pod \"17692add-12e7-421d-8ef4-a5ccb5e850fe\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " Jul 28 20:05:51.824349 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.824266 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-credential-socket\") pod \"17692add-12e7-421d-8ef4-a5ccb5e850fe\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " Jul 28 20:05:51.824349 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.824295 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrmsn\" (UniqueName: \"kubernetes.io/projected/17692add-12e7-421d-8ef4-a5ccb5e850fe-kube-api-access-jrmsn\") pod \"17692add-12e7-421d-8ef4-a5ccb5e850fe\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " Jul 28 20:05:51.824349 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.824339 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17692add-12e7-421d-8ef4-a5ccb5e850fe-istiod-ca-cert\") pod \"17692add-12e7-421d-8ef4-a5ccb5e850fe\" (UID: \"17692add-12e7-421d-8ef4-a5ccb5e850fe\") " Jul 28 20:05:51.825875 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.824998 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "17692add-12e7-421d-8ef4-a5ccb5e850fe" (UID: "17692add-12e7-421d-8ef4-a5ccb5e850fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:05:51.825875 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.825092 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17692add-12e7-421d-8ef4-a5ccb5e850fe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "17692add-12e7-421d-8ef4-a5ccb5e850fe" (UID: "17692add-12e7-421d-8ef4-a5ccb5e850fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:05:51.825875 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.825413 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "17692add-12e7-421d-8ef4-a5ccb5e850fe" (UID: "17692add-12e7-421d-8ef4-a5ccb5e850fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:05:51.825875 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.825440 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "17692add-12e7-421d-8ef4-a5ccb5e850fe" (UID: "17692add-12e7-421d-8ef4-a5ccb5e850fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:05:51.825875 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.825842 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-data" (OuterVolumeSpecName: "istio-data") pod "17692add-12e7-421d-8ef4-a5ccb5e850fe" (UID: "17692add-12e7-421d-8ef4-a5ccb5e850fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:05:51.827919 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.827892 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-token" (OuterVolumeSpecName: "istio-token") pod "17692add-12e7-421d-8ef4-a5ccb5e850fe" (UID: "17692add-12e7-421d-8ef4-a5ccb5e850fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:05:51.829798 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.829746 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "17692add-12e7-421d-8ef4-a5ccb5e850fe" (UID: "17692add-12e7-421d-8ef4-a5ccb5e850fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:05:51.830262 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.830237 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17692add-12e7-421d-8ef4-a5ccb5e850fe-kube-api-access-jrmsn" (OuterVolumeSpecName: "kube-api-access-jrmsn") pod "17692add-12e7-421d-8ef4-a5ccb5e850fe" (UID: "17692add-12e7-421d-8ef4-a5ccb5e850fe"). InnerVolumeSpecName "kube-api-access-jrmsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:05:51.830441 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.830403 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "17692add-12e7-421d-8ef4-a5ccb5e850fe" (UID: "17692add-12e7-421d-8ef4-a5ccb5e850fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 20:05:51.926409 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.926319 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17692add-12e7-421d-8ef4-a5ccb5e850fe-istiod-ca-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:05:51.926409 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.926354 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-workload-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:05:51.926409 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.926369 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-envoy\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:05:51.926409 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.926382 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-podinfo\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:05:51.926409 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.926396 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-token\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:05:51.926409 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.926410 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-istio-data\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:05:51.926839 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.926422 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-workload-certs\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:05:51.926839 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.926435 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17692add-12e7-421d-8ef4-a5ccb5e850fe-credential-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:05:51.926839 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:51.926449 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrmsn\" (UniqueName: \"kubernetes.io/projected/17692add-12e7-421d-8ef4-a5ccb5e850fe-kube-api-access-jrmsn\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:05:52.279211 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:52.279115 2564 generic.go:356] "Generic (PLEG): container finished" podID="17692add-12e7-421d-8ef4-a5ccb5e850fe" containerID="e24eecee4674868adca25b41bcad6cc15eb505d279bd98a18f7e3e81dcffa872" exitCode=0 Jul 28 20:05:52.279211 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:52.279179 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" Jul 28 20:05:52.279211 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:52.279199 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" event={"ID":"17692add-12e7-421d-8ef4-a5ccb5e850fe","Type":"ContainerDied","Data":"e24eecee4674868adca25b41bcad6cc15eb505d279bd98a18f7e3e81dcffa872"} Jul 28 20:05:52.279471 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:52.279269 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m" event={"ID":"17692add-12e7-421d-8ef4-a5ccb5e850fe","Type":"ContainerDied","Data":"4b5b2cbb28951249e2b53c547a4f60819878b580f35359bd13359b92b91a3daf"} Jul 28 20:05:52.279471 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:52.279299 2564 scope.go:117] "RemoveContainer" containerID="e24eecee4674868adca25b41bcad6cc15eb505d279bd98a18f7e3e81dcffa872" Jul 28 20:05:52.302847 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:52.302819 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m"] Jul 28 20:05:52.309679 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:52.309649 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-81375d1e-openshift-default-6bc99cfc7b-kff2m"] Jul 28 20:05:52.487100 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:52.487078 2564 scope.go:117] "RemoveContainer" containerID="7f1c3d0ff75087a24813b08d716fccd5c268396009b5ab3e23da1ed7e8a71a0e" Jul 28 20:05:52.508443 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:52.508424 2564 scope.go:117] "RemoveContainer" containerID="e24eecee4674868adca25b41bcad6cc15eb505d279bd98a18f7e3e81dcffa872" Jul 28 20:05:52.508822 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:05:52.508794 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e24eecee4674868adca25b41bcad6cc15eb505d279bd98a18f7e3e81dcffa872\": container with ID starting with e24eecee4674868adca25b41bcad6cc15eb505d279bd98a18f7e3e81dcffa872 not found: ID does not exist" containerID="e24eecee4674868adca25b41bcad6cc15eb505d279bd98a18f7e3e81dcffa872" Jul 28 20:05:52.508887 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:52.508838 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e24eecee4674868adca25b41bcad6cc15eb505d279bd98a18f7e3e81dcffa872"} err="failed to get container status \"e24eecee4674868adca25b41bcad6cc15eb505d279bd98a18f7e3e81dcffa872\": rpc error: code = NotFound desc = could not find container \"e24eecee4674868adca25b41bcad6cc15eb505d279bd98a18f7e3e81dcffa872\": container with ID starting with e24eecee4674868adca25b41bcad6cc15eb505d279bd98a18f7e3e81dcffa872 not found: ID does not exist" Jul 28 20:05:52.508887 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:52.508863 2564 scope.go:117] "RemoveContainer" containerID="7f1c3d0ff75087a24813b08d716fccd5c268396009b5ab3e23da1ed7e8a71a0e" Jul 28 20:05:52.509140 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:05:52.509126 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f1c3d0ff75087a24813b08d716fccd5c268396009b5ab3e23da1ed7e8a71a0e\": container with ID starting with 7f1c3d0ff75087a24813b08d716fccd5c268396009b5ab3e23da1ed7e8a71a0e not found: ID does not exist" containerID="7f1c3d0ff75087a24813b08d716fccd5c268396009b5ab3e23da1ed7e8a71a0e" Jul 28 20:05:52.509184 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:52.509143 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f1c3d0ff75087a24813b08d716fccd5c268396009b5ab3e23da1ed7e8a71a0e"} err="failed to get container status \"7f1c3d0ff75087a24813b08d716fccd5c268396009b5ab3e23da1ed7e8a71a0e\": rpc error: code = NotFound desc = could not find container \"7f1c3d0ff75087a24813b08d716fccd5c268396009b5ab3e23da1ed7e8a71a0e\": container with ID starting with 7f1c3d0ff75087a24813b08d716fccd5c268396009b5ab3e23da1ed7e8a71a0e not found: ID does not exist" Jul 28 20:05:52.918626 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:52.918589 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17692add-12e7-421d-8ef4-a5ccb5e850fe" path="/var/lib/kubelet/pods/17692add-12e7-421d-8ef4-a5ccb5e850fe/volumes" Jul 28 20:05:53.286676 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:53.286571 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" event={"ID":"fad77ecb-28c4-4bbc-8c5b-cf4596239b0b","Type":"ContainerStarted","Data":"56107f1909e99b8a3d254c648ed3216ee2cb2201293b72f9e1bccbfc371fe6f5"} Jul 28 20:05:53.286846 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:53.286712 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" Jul 28 20:05:53.302742 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:05:53.302693 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" podStartSLOduration=1.604657595 podStartE2EDuration="3.302677562s" totalImagesPullingTime="1.698019967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:05:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:05:53.301796904 +0000 UTC m=+1749.009008798" watchObservedRunningTime="2026-07-28 20:05:53.302677562 +0000 UTC m=+1749.009889454" Jul 28 20:06:04.293880 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:04.293815 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" podUID="fad77ecb-28c4-4bbc-8c5b-cf4596239b0b" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.37:9004: connect: connection refused" Jul 28 20:06:14.293789 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:14.293759 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" Jul 28 20:06:14.543966 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:14.543882 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r"] Jul 28 20:06:14.544326 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:14.544313 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17692add-12e7-421d-8ef4-a5ccb5e850fe" containerName="istio-proxy" Jul 28 20:06:14.544371 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:14.544328 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="17692add-12e7-421d-8ef4-a5ccb5e850fe" containerName="istio-proxy" Jul 28 20:06:14.544407 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:14.544401 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="17692add-12e7-421d-8ef4-a5ccb5e850fe" containerName="istio-proxy" Jul 28 20:06:14.544441 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:14.544412 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="17692add-12e7-421d-8ef4-a5ccb5e850fe" containerName="istio-proxy" Jul 28 20:06:14.550254 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:14.550235 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r" Jul 28 20:06:14.553535 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:14.553511 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7j5xk\"" Jul 28 20:06:14.553688 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:14.553558 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 28 20:06:14.556264 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:14.556233 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r"] Jul 28 20:06:14.640866 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:14.640822 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r\" (UID: \"2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r" Jul 28 20:06:14.742043 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:14.742005 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r\" (UID: \"2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r" Jul 28 20:06:14.742627 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:14.742605 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r\" (UID: \"2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r" Jul 28 20:06:14.861964 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:14.861882 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r" Jul 28 20:06:14.992490 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:14.992465 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r"] Jul 28 20:06:14.994558 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:06:14.994534 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dce2b11_32e7_4e5c_97cf_ea8ce2acf35b.slice/crio-8cd20c1b29cc79573b6596c64378e99207b7aaeb69d427af37954e06e0eefaa3 WatchSource:0}: Error finding container 8cd20c1b29cc79573b6596c64378e99207b7aaeb69d427af37954e06e0eefaa3: Status 404 returned error can't find the container with id 8cd20c1b29cc79573b6596c64378e99207b7aaeb69d427af37954e06e0eefaa3 Jul 28 20:06:15.394890 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:15.394854 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r" event={"ID":"2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b","Type":"ContainerStarted","Data":"8cd20c1b29cc79573b6596c64378e99207b7aaeb69d427af37954e06e0eefaa3"} Jul 28 20:06:18.412992 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:18.412954 2564 generic.go:356] "Generic (PLEG): container finished" podID="2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b" containerID="be8587e72477f3f276882e72a93bbace964e59841ea77a2d4ab501d65cb469b2" exitCode=0 Jul 28 20:06:18.413391 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:18.413037 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r" event={"ID":"2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b","Type":"ContainerDied","Data":"be8587e72477f3f276882e72a93bbace964e59841ea77a2d4ab501d65cb469b2"} Jul 28 20:06:19.549824 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:19.549792 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r" Jul 28 20:06:19.584942 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:19.584917 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b-maas-api-service-ca\") pod \"2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b\" (UID: \"2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b\") " Jul 28 20:06:19.585263 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:19.585241 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b" (UID: "2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:06:19.686135 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:19.686056 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b-maas-api-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:06:20.424390 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:20.424360 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r" Jul 28 20:06:20.424568 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:20.424367 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r" event={"ID":"2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b","Type":"ContainerDied","Data":"8cd20c1b29cc79573b6596c64378e99207b7aaeb69d427af37954e06e0eefaa3"} Jul 28 20:06:20.424568 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:20.424478 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8cd20c1b29cc79573b6596c64378e99207b7aaeb69d427af37954e06e0eefaa3" Jul 28 20:06:30.287971 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:30.287935 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z"] Jul 28 20:06:30.288394 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:30.288197 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" podUID="fad77ecb-28c4-4bbc-8c5b-cf4596239b0b" containerName="payload-pre-processing" containerID="cri-o://56107f1909e99b8a3d254c648ed3216ee2cb2201293b72f9e1bccbfc371fe6f5" gracePeriod=30 Jul 28 20:06:30.433037 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:30.432992 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r"] Jul 28 20:06:30.436013 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:30.435981 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bdcbbca7-dnd7r"] Jul 28 20:06:30.475355 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:30.475320 2564 generic.go:356] "Generic (PLEG): container finished" podID="fad77ecb-28c4-4bbc-8c5b-cf4596239b0b" containerID="56107f1909e99b8a3d254c648ed3216ee2cb2201293b72f9e1bccbfc371fe6f5" exitCode=0 Jul 28 20:06:30.475495 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:30.475386 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" event={"ID":"fad77ecb-28c4-4bbc-8c5b-cf4596239b0b","Type":"ContainerDied","Data":"56107f1909e99b8a3d254c648ed3216ee2cb2201293b72f9e1bccbfc371fe6f5"} Jul 28 20:06:30.553327 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:30.553296 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" Jul 28 20:06:30.586980 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:30.586953 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lp4fr\" (UniqueName: \"kubernetes.io/projected/fad77ecb-28c4-4bbc-8c5b-cf4596239b0b-kube-api-access-lp4fr\") pod \"fad77ecb-28c4-4bbc-8c5b-cf4596239b0b\" (UID: \"fad77ecb-28c4-4bbc-8c5b-cf4596239b0b\") " Jul 28 20:06:30.587154 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:30.587001 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fad77ecb-28c4-4bbc-8c5b-cf4596239b0b-plugins-config-volume\") pod \"fad77ecb-28c4-4bbc-8c5b-cf4596239b0b\" (UID: \"fad77ecb-28c4-4bbc-8c5b-cf4596239b0b\") " Jul 28 20:06:30.587417 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:30.587390 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fad77ecb-28c4-4bbc-8c5b-cf4596239b0b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fad77ecb-28c4-4bbc-8c5b-cf4596239b0b" (UID: "fad77ecb-28c4-4bbc-8c5b-cf4596239b0b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:06:30.589091 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:30.589069 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fad77ecb-28c4-4bbc-8c5b-cf4596239b0b-kube-api-access-lp4fr" (OuterVolumeSpecName: "kube-api-access-lp4fr") pod "fad77ecb-28c4-4bbc-8c5b-cf4596239b0b" (UID: "fad77ecb-28c4-4bbc-8c5b-cf4596239b0b"). InnerVolumeSpecName "kube-api-access-lp4fr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:06:30.687919 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:30.687886 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lp4fr\" (UniqueName: \"kubernetes.io/projected/fad77ecb-28c4-4bbc-8c5b-cf4596239b0b-kube-api-access-lp4fr\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:06:30.687919 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:30.687914 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fad77ecb-28c4-4bbc-8c5b-cf4596239b0b-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:06:30.919242 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:30.919160 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b" path="/var/lib/kubelet/pods/2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b/volumes" Jul 28 20:06:31.360650 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.360609 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449"] Jul 28 20:06:31.361255 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.361237 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad77ecb-28c4-4bbc-8c5b-cf4596239b0b" containerName="payload-pre-processing" Jul 28 20:06:31.361255 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.361256 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad77ecb-28c4-4bbc-8c5b-cf4596239b0b" containerName="payload-pre-processing" Jul 28 20:06:31.361350 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.361271 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b" containerName="revoke-keys" Jul 28 20:06:31.361350 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.361277 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b" containerName="revoke-keys" Jul 28 20:06:31.361350 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.361284 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17692add-12e7-421d-8ef4-a5ccb5e850fe" containerName="istio-proxy" Jul 28 20:06:31.361350 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.361289 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="17692add-12e7-421d-8ef4-a5ccb5e850fe" containerName="istio-proxy" Jul 28 20:06:31.361350 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.361347 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fad77ecb-28c4-4bbc-8c5b-cf4596239b0b" containerName="payload-pre-processing" Jul 28 20:06:31.361509 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.361358 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dce2b11-32e7-4e5c-97cf-ea8ce2acf35b" containerName="revoke-keys" Jul 28 20:06:31.364581 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.364566 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.367322 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.367301 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ae3e83fa-openshift-default-dockercfg-kkkpb\"" Jul 28 20:06:31.377437 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.377413 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449"] Jul 28 20:06:31.394600 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.394569 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn9cn\" (UniqueName: \"kubernetes.io/projected/eb32653e-8048-444c-8d3d-ff3a08a177ff-kube-api-access-gn9cn\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.394763 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.394603 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-podinfo\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.394763 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.394654 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-envoy\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.394882 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.394779 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-credential-socket\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.394882 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.394832 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb32653e-8048-444c-8d3d-ff3a08a177ff-istiod-ca-cert\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.394987 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.394896 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-token\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.394987 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.394960 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-data\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.395088 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.394989 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-workload-certs\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.395088 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.395031 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-workload-socket\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.481975 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.481940 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" Jul 28 20:06:31.482141 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.481945 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z" event={"ID":"fad77ecb-28c4-4bbc-8c5b-cf4596239b0b","Type":"ContainerDied","Data":"bd1be0dbdad13dce314c992f82bd2c8f55c962bd99ce1b01e5ee13f28de71e69"} Jul 28 20:06:31.482141 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.482069 2564 scope.go:117] "RemoveContainer" containerID="56107f1909e99b8a3d254c648ed3216ee2cb2201293b72f9e1bccbfc371fe6f5" Jul 28 20:06:31.495982 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.495956 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-token\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.496089 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.496031 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-data\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.496089 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.496059 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-workload-certs\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.496089 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.496082 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-workload-socket\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.496255 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.496120 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn9cn\" (UniqueName: \"kubernetes.io/projected/eb32653e-8048-444c-8d3d-ff3a08a177ff-kube-api-access-gn9cn\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.496255 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.496181 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-podinfo\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.496363 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.496277 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-envoy\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.496417 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.496366 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-credential-socket\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.496473 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.496434 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb32653e-8048-444c-8d3d-ff3a08a177ff-istiod-ca-cert\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.496525 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.496488 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-workload-certs\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.496647 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.496611 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-workload-socket\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.496989 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.496967 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-credential-socket\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.497191 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.497141 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-data\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.497789 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.497765 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb32653e-8048-444c-8d3d-ff3a08a177ff-istiod-ca-cert\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.498904 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.498886 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-podinfo\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.499142 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.499125 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-envoy\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.506146 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.506120 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn9cn\" (UniqueName: \"kubernetes.io/projected/eb32653e-8048-444c-8d3d-ff3a08a177ff-kube-api-access-gn9cn\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.506257 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.506243 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-token\") pod \"e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.511095 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.511070 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z"] Jul 28 20:06:31.517985 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.517964 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bdcbbca7-5fb76b7fb8-chw7z"] Jul 28 20:06:31.675821 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.675729 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:31.814959 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.814934 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449"] Jul 28 20:06:31.816704 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:06:31.816675 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb32653e_8048_444c_8d3d_ff3a08a177ff.slice/crio-0194a86aca88b274bbe192c1e2ef379b0e430bdbf1c94a482c81699cfee27794 WatchSource:0}: Error finding container 0194a86aca88b274bbe192c1e2ef379b0e430bdbf1c94a482c81699cfee27794: Status 404 returned error can't find the container with id 0194a86aca88b274bbe192c1e2ef379b0e430bdbf1c94a482c81699cfee27794 Jul 28 20:06:31.818905 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.818875 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:06:31.819012 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.818939 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:06:31.819012 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:31.818968 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:06:32.488461 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:32.488425 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" event={"ID":"eb32653e-8048-444c-8d3d-ff3a08a177ff","Type":"ContainerStarted","Data":"03d7154511ddd1e0c1e20e32b513272a09a7070c836fb77ab7ca8657bf78d8ff"} Jul 28 20:06:32.488995 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:32.488491 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" event={"ID":"eb32653e-8048-444c-8d3d-ff3a08a177ff","Type":"ContainerStarted","Data":"0194a86aca88b274bbe192c1e2ef379b0e430bdbf1c94a482c81699cfee27794"} Jul 28 20:06:32.511260 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:32.511210 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" podStartSLOduration=1.5111933789999998 podStartE2EDuration="1.511193379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:06:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:06:32.508234554 +0000 UTC m=+1788.215446476" watchObservedRunningTime="2026-07-28 20:06:32.511193379 +0000 UTC m=+1788.218405272" Jul 28 20:06:32.676589 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:32.676556 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:32.681593 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:32.681570 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:32.919728 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:32.919697 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fad77ecb-28c4-4bbc-8c5b-cf4596239b0b" path="/var/lib/kubelet/pods/fad77ecb-28c4-4bbc-8c5b-cf4596239b0b/volumes" Jul 28 20:06:33.494813 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:33.494782 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:33.495841 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:33.495822 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:34.728297 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:34.728265 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n"] Jul 28 20:06:34.732178 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:34.732157 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" Jul 28 20:06:34.734837 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:34.734805 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ae3e83fa\"" Jul 28 20:06:34.734837 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:34.734814 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ae3e83fa-dockercfg-dzxx5\"" Jul 28 20:06:34.742736 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:34.742710 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n"] Jul 28 20:06:34.828276 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:34.828237 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bef05d12-6a79-47f6-a675-7601b1506564-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n\" (UID: \"bef05d12-6a79-47f6-a675-7601b1506564\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" Jul 28 20:06:34.828465 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:34.828402 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7kk4\" (UniqueName: \"kubernetes.io/projected/bef05d12-6a79-47f6-a675-7601b1506564-kube-api-access-c7kk4\") pod \"payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n\" (UID: \"bef05d12-6a79-47f6-a675-7601b1506564\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" Jul 28 20:06:34.929378 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:34.929332 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bef05d12-6a79-47f6-a675-7601b1506564-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n\" (UID: \"bef05d12-6a79-47f6-a675-7601b1506564\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" Jul 28 20:06:34.929609 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:34.929486 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7kk4\" (UniqueName: \"kubernetes.io/projected/bef05d12-6a79-47f6-a675-7601b1506564-kube-api-access-c7kk4\") pod \"payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n\" (UID: \"bef05d12-6a79-47f6-a675-7601b1506564\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" Jul 28 20:06:34.930115 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:34.930094 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bef05d12-6a79-47f6-a675-7601b1506564-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n\" (UID: \"bef05d12-6a79-47f6-a675-7601b1506564\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" Jul 28 20:06:34.938289 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:34.938258 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7kk4\" (UniqueName: \"kubernetes.io/projected/bef05d12-6a79-47f6-a675-7601b1506564-kube-api-access-c7kk4\") pod \"payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n\" (UID: \"bef05d12-6a79-47f6-a675-7601b1506564\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" Jul 28 20:06:35.044561 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:35.044523 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" Jul 28 20:06:35.178014 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:35.177990 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n"] Jul 28 20:06:35.179982 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:06:35.179955 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbef05d12_6a79_47f6_a675_7601b1506564.slice/crio-c7930b117e2dd480c2e8e98bc622d6d223a92d308c25d04904dae0284cf8c999 WatchSource:0}: Error finding container c7930b117e2dd480c2e8e98bc622d6d223a92d308c25d04904dae0284cf8c999: Status 404 returned error can't find the container with id c7930b117e2dd480c2e8e98bc622d6d223a92d308c25d04904dae0284cf8c999 Jul 28 20:06:35.507167 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:35.507068 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" event={"ID":"bef05d12-6a79-47f6-a675-7601b1506564","Type":"ContainerStarted","Data":"4ca4877034422ef957097c9bd29d8a79062f665563b7ead147d1b8bbe8821e79"} Jul 28 20:06:35.507167 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:35.507133 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" event={"ID":"bef05d12-6a79-47f6-a675-7601b1506564","Type":"ContainerStarted","Data":"c7930b117e2dd480c2e8e98bc622d6d223a92d308c25d04904dae0284cf8c999"} Jul 28 20:06:35.507395 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:35.507289 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" Jul 28 20:06:35.528694 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:35.528626 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" podStartSLOduration=1.528611535 podStartE2EDuration="1.528611535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:06:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:06:35.527381564 +0000 UTC m=+1791.234593469" watchObservedRunningTime="2026-07-28 20:06:35.528611535 +0000 UTC m=+1791.235823428" Jul 28 20:06:46.515318 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:46.515287 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" Jul 28 20:06:48.496494 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:48.496456 2564 patch_prober.go:28] interesting pod/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 28 20:06:48.496980 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:48.496517 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" podUID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 28 20:06:48.603880 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:48.603842 2564 generic.go:356] "Generic (PLEG): container finished" podID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerID="03d7154511ddd1e0c1e20e32b513272a09a7070c836fb77ab7ca8657bf78d8ff" exitCode=0 Jul 28 20:06:48.604063 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:48.603915 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" event={"ID":"eb32653e-8048-444c-8d3d-ff3a08a177ff","Type":"ContainerDied","Data":"03d7154511ddd1e0c1e20e32b513272a09a7070c836fb77ab7ca8657bf78d8ff"} Jul 28 20:06:48.604330 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:48.604314 2564 scope.go:117] "RemoveContainer" containerID="03d7154511ddd1e0c1e20e32b513272a09a7070c836fb77ab7ca8657bf78d8ff" Jul 28 20:06:48.605370 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:48.605344 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:06:48.605450 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:48.605436 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:06:48.605487 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:48.605478 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:06:49.611200 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:49.611167 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" event={"ID":"eb32653e-8048-444c-8d3d-ff3a08a177ff","Type":"ContainerStarted","Data":"92f15d8f515cb02e0f78542fb6f0c80a915c3dbf425cba6dcf74bcd359aceb0f"} Jul 28 20:06:49.676034 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:49.675991 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:49.676322 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:49.676295 2564 patch_prober.go:28] interesting pod/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 28 20:06:49.676374 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:49.676353 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" podUID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 28 20:06:50.676324 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:50.676280 2564 patch_prober.go:28] interesting pod/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 28 20:06:50.676796 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:50.676363 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" podUID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 28 20:06:51.677056 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:51.677011 2564 patch_prober.go:28] interesting pod/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 28 20:06:51.677466 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:51.677076 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" podUID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 28 20:06:52.676391 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:52.676282 2564 patch_prober.go:28] interesting pod/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 28 20:06:52.676391 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:52.676351 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" podUID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 28 20:06:53.677121 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:53.677079 2564 patch_prober.go:28] interesting pod/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 28 20:06:53.677557 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:53.677151 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" podUID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 28 20:06:54.680647 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:54.680602 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:54.681105 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:54.681085 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:06:54.681722 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:06:54.681703 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:07:04.844970 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:04.844924 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n"] Jul 28 20:07:04.845445 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:04.845187 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" podUID="bef05d12-6a79-47f6-a675-7601b1506564" containerName="payload-pre-processing" containerID="cri-o://4ca4877034422ef957097c9bd29d8a79062f665563b7ead147d1b8bbe8821e79" gracePeriod=30 Jul 28 20:07:05.115555 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.115485 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" Jul 28 20:07:05.132498 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.132468 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bef05d12-6a79-47f6-a675-7601b1506564-plugins-config-volume\") pod \"bef05d12-6a79-47f6-a675-7601b1506564\" (UID: \"bef05d12-6a79-47f6-a675-7601b1506564\") " Jul 28 20:07:05.132730 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.132546 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7kk4\" (UniqueName: \"kubernetes.io/projected/bef05d12-6a79-47f6-a675-7601b1506564-kube-api-access-c7kk4\") pod \"bef05d12-6a79-47f6-a675-7601b1506564\" (UID: \"bef05d12-6a79-47f6-a675-7601b1506564\") " Jul 28 20:07:05.132941 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.132911 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bef05d12-6a79-47f6-a675-7601b1506564-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bef05d12-6a79-47f6-a675-7601b1506564" (UID: "bef05d12-6a79-47f6-a675-7601b1506564"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:07:05.135738 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.135703 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bef05d12-6a79-47f6-a675-7601b1506564-kube-api-access-c7kk4" (OuterVolumeSpecName: "kube-api-access-c7kk4") pod "bef05d12-6a79-47f6-a675-7601b1506564" (UID: "bef05d12-6a79-47f6-a675-7601b1506564"). InnerVolumeSpecName "kube-api-access-c7kk4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:07:05.234123 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.234091 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bef05d12-6a79-47f6-a675-7601b1506564-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:05.234123 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.234119 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7kk4\" (UniqueName: \"kubernetes.io/projected/bef05d12-6a79-47f6-a675-7601b1506564-kube-api-access-c7kk4\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:05.696660 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.696605 2564 generic.go:356] "Generic (PLEG): container finished" podID="bef05d12-6a79-47f6-a675-7601b1506564" containerID="4ca4877034422ef957097c9bd29d8a79062f665563b7ead147d1b8bbe8821e79" exitCode=0 Jul 28 20:07:05.696809 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.696668 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" Jul 28 20:07:05.696809 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.696690 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" event={"ID":"bef05d12-6a79-47f6-a675-7601b1506564","Type":"ContainerDied","Data":"4ca4877034422ef957097c9bd29d8a79062f665563b7ead147d1b8bbe8821e79"} Jul 28 20:07:05.696809 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.696744 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n" event={"ID":"bef05d12-6a79-47f6-a675-7601b1506564","Type":"ContainerDied","Data":"c7930b117e2dd480c2e8e98bc622d6d223a92d308c25d04904dae0284cf8c999"} Jul 28 20:07:05.696809 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.696767 2564 scope.go:117] "RemoveContainer" containerID="4ca4877034422ef957097c9bd29d8a79062f665563b7ead147d1b8bbe8821e79" Jul 28 20:07:05.706013 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.705996 2564 scope.go:117] "RemoveContainer" containerID="4ca4877034422ef957097c9bd29d8a79062f665563b7ead147d1b8bbe8821e79" Jul 28 20:07:05.706294 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:07:05.706276 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ca4877034422ef957097c9bd29d8a79062f665563b7ead147d1b8bbe8821e79\": container with ID starting with 4ca4877034422ef957097c9bd29d8a79062f665563b7ead147d1b8bbe8821e79 not found: ID does not exist" containerID="4ca4877034422ef957097c9bd29d8a79062f665563b7ead147d1b8bbe8821e79" Jul 28 20:07:05.706338 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.706303 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ca4877034422ef957097c9bd29d8a79062f665563b7ead147d1b8bbe8821e79"} err="failed to get container status \"4ca4877034422ef957097c9bd29d8a79062f665563b7ead147d1b8bbe8821e79\": rpc error: code = NotFound desc = could not find container \"4ca4877034422ef957097c9bd29d8a79062f665563b7ead147d1b8bbe8821e79\": container with ID starting with 4ca4877034422ef957097c9bd29d8a79062f665563b7ead147d1b8bbe8821e79 not found: ID does not exist" Jul 28 20:07:05.718730 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.718704 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n"] Jul 28 20:07:05.722918 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:05.722882 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae3e83fa-59dbf887d8-xcz2n"] Jul 28 20:07:06.061804 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.061770 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449"] Jul 28 20:07:06.062246 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.061980 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" podUID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerName="istio-proxy" containerID="cri-o://92f15d8f515cb02e0f78542fb6f0c80a915c3dbf425cba6dcf74bcd359aceb0f" gracePeriod=30 Jul 28 20:07:06.798386 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.798351 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b"] Jul 28 20:07:06.798768 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.798755 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bef05d12-6a79-47f6-a675-7601b1506564" containerName="payload-pre-processing" Jul 28 20:07:06.798819 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.798770 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bef05d12-6a79-47f6-a675-7601b1506564" containerName="payload-pre-processing" Jul 28 20:07:06.798855 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.798838 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bef05d12-6a79-47f6-a675-7601b1506564" containerName="payload-pre-processing" Jul 28 20:07:06.803209 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.803193 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.806670 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.806619 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-19b71b42-openshift-default-dockercfg-ssbdz\"" Jul 28 20:07:06.818720 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.818690 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b"] Jul 28 20:07:06.850714 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.850678 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-podinfo\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.850901 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.850729 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-workload-certs\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.850901 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.850784 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-workload-socket\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.850901 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.850817 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istiod-ca-cert\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.850901 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.850859 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-envoy\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.850901 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.850883 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-data\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.851086 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.850918 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-credential-socket\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.851086 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.851002 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-token\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.851086 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.851040 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn8f8\" (UniqueName: \"kubernetes.io/projected/3b1da88a-a253-4948-85a7-20e4fb21b8d7-kube-api-access-jn8f8\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.921002 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.920968 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bef05d12-6a79-47f6-a675-7601b1506564" path="/var/lib/kubelet/pods/bef05d12-6a79-47f6-a675-7601b1506564/volumes" Jul 28 20:07:06.952275 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.952239 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-token\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.952471 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.952299 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn8f8\" (UniqueName: \"kubernetes.io/projected/3b1da88a-a253-4948-85a7-20e4fb21b8d7-kube-api-access-jn8f8\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.952471 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.952339 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-podinfo\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.952471 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.952373 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-workload-certs\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.952471 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.952403 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-workload-socket\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.952471 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.952425 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istiod-ca-cert\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.952837 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.952483 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-envoy\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.952837 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.952533 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-data\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.952837 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.952621 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-credential-socket\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.952998 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.952834 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-workload-certs\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.953055 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.953004 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-credential-socket\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.953163 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.953088 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-data\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.953163 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.953133 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-workload-socket\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.953245 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.953200 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istiod-ca-cert\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.954865 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.954843 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-envoy\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.955082 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.955063 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-podinfo\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.961010 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.960974 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-token\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:06.961431 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:06.961408 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn8f8\" (UniqueName: \"kubernetes.io/projected/3b1da88a-a253-4948-85a7-20e4fb21b8d7-kube-api-access-jn8f8\") pod \"e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:07.114659 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:07.114532 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:07.259707 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:07.259676 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b"] Jul 28 20:07:07.261308 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:07:07.261280 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b1da88a_a253_4948_85a7_20e4fb21b8d7.slice/crio-58eec1d307a4ef50508076cd03b58e2cf1314f590c038af00e75e1c0af61b508 WatchSource:0}: Error finding container 58eec1d307a4ef50508076cd03b58e2cf1314f590c038af00e75e1c0af61b508: Status 404 returned error can't find the container with id 58eec1d307a4ef50508076cd03b58e2cf1314f590c038af00e75e1c0af61b508 Jul 28 20:07:07.263728 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:07.263700 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:07:07.263786 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:07.263765 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:07:07.263828 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:07.263794 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:07:07.712236 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:07.712201 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" event={"ID":"3b1da88a-a253-4948-85a7-20e4fb21b8d7","Type":"ContainerStarted","Data":"fa07b2bf5114b5bb6232f1b49cc8f0fd569a8d8a55b93867aeab423681401729"} Jul 28 20:07:07.712428 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:07.712255 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" event={"ID":"3b1da88a-a253-4948-85a7-20e4fb21b8d7","Type":"ContainerStarted","Data":"58eec1d307a4ef50508076cd03b58e2cf1314f590c038af00e75e1c0af61b508"} Jul 28 20:07:07.739054 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:07.739000 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" podStartSLOduration=1.738982399 podStartE2EDuration="1.738982399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:07:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:07:07.736830905 +0000 UTC m=+1823.444042843" watchObservedRunningTime="2026-07-28 20:07:07.738982399 +0000 UTC m=+1823.446194291" Jul 28 20:07:08.114892 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:08.114849 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:08.119839 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:08.119812 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:08.717384 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:08.717353 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:08.718356 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:08.718336 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:09.683521 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:09.683481 2564 patch_prober.go:28] interesting pod/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 28 20:07:09.683941 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:09.683542 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" podUID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 20:07:09.866606 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:09.866571 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk"] Jul 28 20:07:09.870224 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:09.870207 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" Jul 28 20:07:09.873106 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:09.873087 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-19b71b42\"" Jul 28 20:07:09.880752 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:09.880721 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk"] Jul 28 20:07:09.984170 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:09.984066 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k27pt\" (UniqueName: \"kubernetes.io/projected/67e388f5-a6b6-4824-8095-46b72ea749b2-kube-api-access-k27pt\") pod \"payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk\" (UID: \"67e388f5-a6b6-4824-8095-46b72ea749b2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" Jul 28 20:07:09.984354 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:09.984232 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67e388f5-a6b6-4824-8095-46b72ea749b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk\" (UID: \"67e388f5-a6b6-4824-8095-46b72ea749b2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" Jul 28 20:07:10.085096 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:10.085053 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67e388f5-a6b6-4824-8095-46b72ea749b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk\" (UID: \"67e388f5-a6b6-4824-8095-46b72ea749b2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" Jul 28 20:07:10.085290 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:10.085138 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k27pt\" (UniqueName: \"kubernetes.io/projected/67e388f5-a6b6-4824-8095-46b72ea749b2-kube-api-access-k27pt\") pod \"payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk\" (UID: \"67e388f5-a6b6-4824-8095-46b72ea749b2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" Jul 28 20:07:10.085793 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:10.085772 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67e388f5-a6b6-4824-8095-46b72ea749b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk\" (UID: \"67e388f5-a6b6-4824-8095-46b72ea749b2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" Jul 28 20:07:10.094509 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:10.094472 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k27pt\" (UniqueName: \"kubernetes.io/projected/67e388f5-a6b6-4824-8095-46b72ea749b2-kube-api-access-k27pt\") pod \"payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk\" (UID: \"67e388f5-a6b6-4824-8095-46b72ea749b2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" Jul 28 20:07:10.183197 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:10.183150 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" Jul 28 20:07:10.322265 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:10.322238 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk"] Jul 28 20:07:10.324492 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:07:10.324456 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e388f5_a6b6_4824_8095_46b72ea749b2.slice/crio-71336c3a59e1b005a58775f84b394fb0d1d8c2cc46ca1d567c29b97b2e564755 WatchSource:0}: Error finding container 71336c3a59e1b005a58775f84b394fb0d1d8c2cc46ca1d567c29b97b2e564755: Status 404 returned error can't find the container with id 71336c3a59e1b005a58775f84b394fb0d1d8c2cc46ca1d567c29b97b2e564755 Jul 28 20:07:10.728803 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:10.728760 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" event={"ID":"67e388f5-a6b6-4824-8095-46b72ea749b2","Type":"ContainerStarted","Data":"345a3140f55830aaf4fc74f57dcc70ab13f6673699d34c572ea755204ef4762d"} Jul 28 20:07:10.729188 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:10.728814 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" event={"ID":"67e388f5-a6b6-4824-8095-46b72ea749b2","Type":"ContainerStarted","Data":"71336c3a59e1b005a58775f84b394fb0d1d8c2cc46ca1d567c29b97b2e564755"} Jul 28 20:07:10.729188 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:10.728855 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" Jul 28 20:07:10.751285 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:10.751221 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" podStartSLOduration=1.751200861 podStartE2EDuration="1.751200861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:07:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:07:10.747297683 +0000 UTC m=+1826.454509591" watchObservedRunningTime="2026-07-28 20:07:10.751200861 +0000 UTC m=+1826.458412756" Jul 28 20:07:11.315574 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.315547 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:07:11.400365 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.400276 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-credential-socket\") pod \"eb32653e-8048-444c-8d3d-ff3a08a177ff\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " Jul 28 20:07:11.400365 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.400315 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-podinfo\") pod \"eb32653e-8048-444c-8d3d-ff3a08a177ff\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " Jul 28 20:07:11.400365 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.400352 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-token\") pod \"eb32653e-8048-444c-8d3d-ff3a08a177ff\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " Jul 28 20:07:11.400674 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.400383 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb32653e-8048-444c-8d3d-ff3a08a177ff-istiod-ca-cert\") pod \"eb32653e-8048-444c-8d3d-ff3a08a177ff\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " Jul 28 20:07:11.400674 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.400423 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-data\") pod \"eb32653e-8048-444c-8d3d-ff3a08a177ff\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " Jul 28 20:07:11.400674 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.400439 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-envoy\") pod \"eb32653e-8048-444c-8d3d-ff3a08a177ff\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " Jul 28 20:07:11.400674 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.400489 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn9cn\" (UniqueName: \"kubernetes.io/projected/eb32653e-8048-444c-8d3d-ff3a08a177ff-kube-api-access-gn9cn\") pod \"eb32653e-8048-444c-8d3d-ff3a08a177ff\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " Jul 28 20:07:11.400674 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.400544 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-workload-socket\") pod \"eb32653e-8048-444c-8d3d-ff3a08a177ff\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " Jul 28 20:07:11.400674 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.400567 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-workload-certs\") pod \"eb32653e-8048-444c-8d3d-ff3a08a177ff\" (UID: \"eb32653e-8048-444c-8d3d-ff3a08a177ff\") " Jul 28 20:07:11.400674 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.400618 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eb32653e-8048-444c-8d3d-ff3a08a177ff" (UID: "eb32653e-8048-444c-8d3d-ff3a08a177ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:07:11.401051 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.400894 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eb32653e-8048-444c-8d3d-ff3a08a177ff" (UID: "eb32653e-8048-444c-8d3d-ff3a08a177ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:07:11.401051 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.400911 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-credential-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:11.401051 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.401013 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb32653e-8048-444c-8d3d-ff3a08a177ff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eb32653e-8048-444c-8d3d-ff3a08a177ff" (UID: "eb32653e-8048-444c-8d3d-ff3a08a177ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:07:11.401199 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.401114 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eb32653e-8048-444c-8d3d-ff3a08a177ff" (UID: "eb32653e-8048-444c-8d3d-ff3a08a177ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:07:11.401594 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.401569 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-data" (OuterVolumeSpecName: "istio-data") pod "eb32653e-8048-444c-8d3d-ff3a08a177ff" (UID: "eb32653e-8048-444c-8d3d-ff3a08a177ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:07:11.402803 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.402762 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eb32653e-8048-444c-8d3d-ff3a08a177ff" (UID: "eb32653e-8048-444c-8d3d-ff3a08a177ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 20:07:11.402922 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.402869 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-token" (OuterVolumeSpecName: "istio-token") pod "eb32653e-8048-444c-8d3d-ff3a08a177ff" (UID: "eb32653e-8048-444c-8d3d-ff3a08a177ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:07:11.403066 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.403048 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eb32653e-8048-444c-8d3d-ff3a08a177ff" (UID: "eb32653e-8048-444c-8d3d-ff3a08a177ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:07:11.403158 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.403138 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb32653e-8048-444c-8d3d-ff3a08a177ff-kube-api-access-gn9cn" (OuterVolumeSpecName: "kube-api-access-gn9cn") pod "eb32653e-8048-444c-8d3d-ff3a08a177ff" (UID: "eb32653e-8048-444c-8d3d-ff3a08a177ff"). InnerVolumeSpecName "kube-api-access-gn9cn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:07:11.502299 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.502262 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-token\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:11.502299 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.502292 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb32653e-8048-444c-8d3d-ff3a08a177ff-istiod-ca-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:11.502299 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.502301 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-data\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:11.502299 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.502308 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-envoy\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:11.502572 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.502317 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn9cn\" (UniqueName: \"kubernetes.io/projected/eb32653e-8048-444c-8d3d-ff3a08a177ff-kube-api-access-gn9cn\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:11.502572 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.502328 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-workload-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:11.502572 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.502336 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb32653e-8048-444c-8d3d-ff3a08a177ff-workload-certs\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:11.502572 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.502343 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb32653e-8048-444c-8d3d-ff3a08a177ff-istio-podinfo\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:11.741880 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.741774 2564 generic.go:356] "Generic (PLEG): container finished" podID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerID="92f15d8f515cb02e0f78542fb6f0c80a915c3dbf425cba6dcf74bcd359aceb0f" exitCode=0 Jul 28 20:07:11.741880 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.741850 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" Jul 28 20:07:11.741880 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.741861 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" event={"ID":"eb32653e-8048-444c-8d3d-ff3a08a177ff","Type":"ContainerDied","Data":"92f15d8f515cb02e0f78542fb6f0c80a915c3dbf425cba6dcf74bcd359aceb0f"} Jul 28 20:07:11.742426 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.741946 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449" event={"ID":"eb32653e-8048-444c-8d3d-ff3a08a177ff","Type":"ContainerDied","Data":"0194a86aca88b274bbe192c1e2ef379b0e430bdbf1c94a482c81699cfee27794"} Jul 28 20:07:11.742426 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.741970 2564 scope.go:117] "RemoveContainer" containerID="92f15d8f515cb02e0f78542fb6f0c80a915c3dbf425cba6dcf74bcd359aceb0f" Jul 28 20:07:11.751976 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.751957 2564 scope.go:117] "RemoveContainer" containerID="03d7154511ddd1e0c1e20e32b513272a09a7070c836fb77ab7ca8657bf78d8ff" Jul 28 20:07:11.760111 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.760092 2564 scope.go:117] "RemoveContainer" containerID="92f15d8f515cb02e0f78542fb6f0c80a915c3dbf425cba6dcf74bcd359aceb0f" Jul 28 20:07:11.760388 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:07:11.760366 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92f15d8f515cb02e0f78542fb6f0c80a915c3dbf425cba6dcf74bcd359aceb0f\": container with ID starting with 92f15d8f515cb02e0f78542fb6f0c80a915c3dbf425cba6dcf74bcd359aceb0f not found: ID does not exist" containerID="92f15d8f515cb02e0f78542fb6f0c80a915c3dbf425cba6dcf74bcd359aceb0f" Jul 28 20:07:11.760445 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.760395 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92f15d8f515cb02e0f78542fb6f0c80a915c3dbf425cba6dcf74bcd359aceb0f"} err="failed to get container status \"92f15d8f515cb02e0f78542fb6f0c80a915c3dbf425cba6dcf74bcd359aceb0f\": rpc error: code = NotFound desc = could not find container \"92f15d8f515cb02e0f78542fb6f0c80a915c3dbf425cba6dcf74bcd359aceb0f\": container with ID starting with 92f15d8f515cb02e0f78542fb6f0c80a915c3dbf425cba6dcf74bcd359aceb0f not found: ID does not exist" Jul 28 20:07:11.760445 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.760412 2564 scope.go:117] "RemoveContainer" containerID="03d7154511ddd1e0c1e20e32b513272a09a7070c836fb77ab7ca8657bf78d8ff" Jul 28 20:07:11.760659 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:07:11.760623 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03d7154511ddd1e0c1e20e32b513272a09a7070c836fb77ab7ca8657bf78d8ff\": container with ID starting with 03d7154511ddd1e0c1e20e32b513272a09a7070c836fb77ab7ca8657bf78d8ff not found: ID does not exist" containerID="03d7154511ddd1e0c1e20e32b513272a09a7070c836fb77ab7ca8657bf78d8ff" Jul 28 20:07:11.760770 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.760666 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03d7154511ddd1e0c1e20e32b513272a09a7070c836fb77ab7ca8657bf78d8ff"} err="failed to get container status \"03d7154511ddd1e0c1e20e32b513272a09a7070c836fb77ab7ca8657bf78d8ff\": rpc error: code = NotFound desc = could not find container \"03d7154511ddd1e0c1e20e32b513272a09a7070c836fb77ab7ca8657bf78d8ff\": container with ID starting with 03d7154511ddd1e0c1e20e32b513272a09a7070c836fb77ab7ca8657bf78d8ff not found: ID does not exist" Jul 28 20:07:11.764586 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.764560 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449"] Jul 28 20:07:11.772424 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:11.772402 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ae3e83fa-openshift-default-d64f5cd76-tb449"] Jul 28 20:07:12.918414 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:12.918381 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb32653e-8048-444c-8d3d-ff3a08a177ff" path="/var/lib/kubelet/pods/eb32653e-8048-444c-8d3d-ff3a08a177ff/volumes" Jul 28 20:07:21.744337 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:21.744305 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" Jul 28 20:07:22.801227 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:22.801192 2564 generic.go:356] "Generic (PLEG): container finished" podID="3b1da88a-a253-4948-85a7-20e4fb21b8d7" containerID="fa07b2bf5114b5bb6232f1b49cc8f0fd569a8d8a55b93867aeab423681401729" exitCode=0 Jul 28 20:07:22.801728 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:22.801242 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" event={"ID":"3b1da88a-a253-4948-85a7-20e4fb21b8d7","Type":"ContainerDied","Data":"fa07b2bf5114b5bb6232f1b49cc8f0fd569a8d8a55b93867aeab423681401729"} Jul 28 20:07:22.801728 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:22.801664 2564 scope.go:117] "RemoveContainer" containerID="fa07b2bf5114b5bb6232f1b49cc8f0fd569a8d8a55b93867aeab423681401729" Jul 28 20:07:22.802712 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:22.802681 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:07:22.802812 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:22.802751 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:07:22.802812 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:22.802785 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:07:23.717705 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:23.717577 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:23.808163 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:23.808125 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" event={"ID":"3b1da88a-a253-4948-85a7-20e4fb21b8d7","Type":"ContainerStarted","Data":"f89bc3e55b2543c97761b8ada2e73604e32d704666ae7eb3e57eb47be18b0027"} Jul 28 20:07:24.114739 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:24.114698 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:24.115951 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:24.115923 2564 patch_prober.go:28] interesting pod/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 28 20:07:24.116072 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:24.115975 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" podUID="3b1da88a-a253-4948-85a7-20e4fb21b8d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 28 20:07:25.115235 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:25.115198 2564 patch_prober.go:28] interesting pod/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 28 20:07:25.115671 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:25.115285 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" podUID="3b1da88a-a253-4948-85a7-20e4fb21b8d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 28 20:07:26.115455 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:26.115420 2564 patch_prober.go:28] interesting pod/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 28 20:07:26.115868 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:26.115488 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" podUID="3b1da88a-a253-4948-85a7-20e4fb21b8d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 28 20:07:27.115828 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:27.115793 2564 patch_prober.go:28] interesting pod/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 28 20:07:27.116230 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:27.115850 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" podUID="3b1da88a-a253-4948-85a7-20e4fb21b8d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 28 20:07:28.119847 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:28.119816 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:28.120242 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:28.120099 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:28.120842 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:28.120819 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:39.796503 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:39.796467 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk"] Jul 28 20:07:39.796992 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:39.796751 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" podUID="67e388f5-a6b6-4824-8095-46b72ea749b2" containerName="payload-pre-processing" containerID="cri-o://345a3140f55830aaf4fc74f57dcc70ab13f6673699d34c572ea755204ef4762d" gracePeriod=30 Jul 28 20:07:40.050025 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.049945 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b"] Jul 28 20:07:40.050220 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.050173 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" podUID="3b1da88a-a253-4948-85a7-20e4fb21b8d7" containerName="istio-proxy" containerID="cri-o://f89bc3e55b2543c97761b8ada2e73604e32d704666ae7eb3e57eb47be18b0027" gracePeriod=30 Jul 28 20:07:40.081939 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.081908 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" Jul 28 20:07:40.208262 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.208223 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67e388f5-a6b6-4824-8095-46b72ea749b2-plugins-config-volume\") pod \"67e388f5-a6b6-4824-8095-46b72ea749b2\" (UID: \"67e388f5-a6b6-4824-8095-46b72ea749b2\") " Jul 28 20:07:40.208262 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.208261 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k27pt\" (UniqueName: \"kubernetes.io/projected/67e388f5-a6b6-4824-8095-46b72ea749b2-kube-api-access-k27pt\") pod \"67e388f5-a6b6-4824-8095-46b72ea749b2\" (UID: \"67e388f5-a6b6-4824-8095-46b72ea749b2\") " Jul 28 20:07:40.208599 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.208570 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67e388f5-a6b6-4824-8095-46b72ea749b2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "67e388f5-a6b6-4824-8095-46b72ea749b2" (UID: "67e388f5-a6b6-4824-8095-46b72ea749b2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:07:40.210404 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.210374 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67e388f5-a6b6-4824-8095-46b72ea749b2-kube-api-access-k27pt" (OuterVolumeSpecName: "kube-api-access-k27pt") pod "67e388f5-a6b6-4824-8095-46b72ea749b2" (UID: "67e388f5-a6b6-4824-8095-46b72ea749b2"). InnerVolumeSpecName "kube-api-access-k27pt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:07:40.309533 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.309447 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67e388f5-a6b6-4824-8095-46b72ea749b2-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:40.309533 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.309481 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k27pt\" (UniqueName: \"kubernetes.io/projected/67e388f5-a6b6-4824-8095-46b72ea749b2-kube-api-access-k27pt\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:40.894198 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.894161 2564 generic.go:356] "Generic (PLEG): container finished" podID="67e388f5-a6b6-4824-8095-46b72ea749b2" containerID="345a3140f55830aaf4fc74f57dcc70ab13f6673699d34c572ea755204ef4762d" exitCode=0 Jul 28 20:07:40.894684 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.894209 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" Jul 28 20:07:40.894684 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.894255 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" event={"ID":"67e388f5-a6b6-4824-8095-46b72ea749b2","Type":"ContainerDied","Data":"345a3140f55830aaf4fc74f57dcc70ab13f6673699d34c572ea755204ef4762d"} Jul 28 20:07:40.894684 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.894303 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk" event={"ID":"67e388f5-a6b6-4824-8095-46b72ea749b2","Type":"ContainerDied","Data":"71336c3a59e1b005a58775f84b394fb0d1d8c2cc46ca1d567c29b97b2e564755"} Jul 28 20:07:40.894684 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.894328 2564 scope.go:117] "RemoveContainer" containerID="345a3140f55830aaf4fc74f57dcc70ab13f6673699d34c572ea755204ef4762d" Jul 28 20:07:40.903378 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.903360 2564 scope.go:117] "RemoveContainer" containerID="345a3140f55830aaf4fc74f57dcc70ab13f6673699d34c572ea755204ef4762d" Jul 28 20:07:40.903666 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:07:40.903630 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"345a3140f55830aaf4fc74f57dcc70ab13f6673699d34c572ea755204ef4762d\": container with ID starting with 345a3140f55830aaf4fc74f57dcc70ab13f6673699d34c572ea755204ef4762d not found: ID does not exist" containerID="345a3140f55830aaf4fc74f57dcc70ab13f6673699d34c572ea755204ef4762d" Jul 28 20:07:40.903711 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.903676 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"345a3140f55830aaf4fc74f57dcc70ab13f6673699d34c572ea755204ef4762d"} err="failed to get container status \"345a3140f55830aaf4fc74f57dcc70ab13f6673699d34c572ea755204ef4762d\": rpc error: code = NotFound desc = could not find container \"345a3140f55830aaf4fc74f57dcc70ab13f6673699d34c572ea755204ef4762d\": container with ID starting with 345a3140f55830aaf4fc74f57dcc70ab13f6673699d34c572ea755204ef4762d not found: ID does not exist" Jul 28 20:07:40.919146 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.919123 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk"] Jul 28 20:07:40.921759 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:40.921736 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-19b71b42-7f456f7d8d-ps6tk"] Jul 28 20:07:42.917786 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:42.917753 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67e388f5-a6b6-4824-8095-46b72ea749b2" path="/var/lib/kubelet/pods/67e388f5-a6b6-4824-8095-46b72ea749b2/volumes" Jul 28 20:07:43.122283 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:43.122247 2564 patch_prober.go:28] interesting pod/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 28 20:07:43.122446 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:43.122303 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" podUID="3b1da88a-a253-4948-85a7-20e4fb21b8d7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 20:07:45.308837 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.308813 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:45.356618 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.356584 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istiod-ca-cert\") pod \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " Jul 28 20:07:45.356788 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.356663 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-token\") pod \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " Jul 28 20:07:45.356788 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.356693 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn8f8\" (UniqueName: \"kubernetes.io/projected/3b1da88a-a253-4948-85a7-20e4fb21b8d7-kube-api-access-jn8f8\") pod \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " Jul 28 20:07:45.356788 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.356716 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-credential-socket\") pod \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " Jul 28 20:07:45.356788 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.356740 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-data\") pod \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " Jul 28 20:07:45.356952 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.356864 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-workload-certs\") pod \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " Jul 28 20:07:45.356952 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.356909 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-envoy\") pod \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " Jul 28 20:07:45.356952 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.356945 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-podinfo\") pod \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " Jul 28 20:07:45.357062 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.356981 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-workload-socket\") pod \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\" (UID: \"3b1da88a-a253-4948-85a7-20e4fb21b8d7\") " Jul 28 20:07:45.357062 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.357040 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b1da88a-a253-4948-85a7-20e4fb21b8d7" (UID: "3b1da88a-a253-4948-85a7-20e4fb21b8d7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:07:45.357062 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.357053 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b1da88a-a253-4948-85a7-20e4fb21b8d7" (UID: "3b1da88a-a253-4948-85a7-20e4fb21b8d7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:07:45.357286 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.357244 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b1da88a-a253-4948-85a7-20e4fb21b8d7" (UID: "3b1da88a-a253-4948-85a7-20e4fb21b8d7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:07:45.357443 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.357423 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istiod-ca-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:45.357607 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.357593 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-credential-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:45.357762 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.357739 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-workload-certs\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:45.357884 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.357431 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b1da88a-a253-4948-85a7-20e4fb21b8d7" (UID: "3b1da88a-a253-4948-85a7-20e4fb21b8d7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:07:45.357884 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.357770 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b1da88a-a253-4948-85a7-20e4fb21b8d7" (UID: "3b1da88a-a253-4948-85a7-20e4fb21b8d7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:07:45.359073 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.359037 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b1da88a-a253-4948-85a7-20e4fb21b8d7" (UID: "3b1da88a-a253-4948-85a7-20e4fb21b8d7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:07:45.359161 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.359098 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b1da88a-a253-4948-85a7-20e4fb21b8d7" (UID: "3b1da88a-a253-4948-85a7-20e4fb21b8d7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:07:45.359216 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.359198 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b1da88a-a253-4948-85a7-20e4fb21b8d7" (UID: "3b1da88a-a253-4948-85a7-20e4fb21b8d7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 20:07:45.359261 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.359211 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b1da88a-a253-4948-85a7-20e4fb21b8d7-kube-api-access-jn8f8" (OuterVolumeSpecName: "kube-api-access-jn8f8") pod "3b1da88a-a253-4948-85a7-20e4fb21b8d7" (UID: "3b1da88a-a253-4948-85a7-20e4fb21b8d7"). InnerVolumeSpecName "kube-api-access-jn8f8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:07:45.458907 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.458877 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-token\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:45.458907 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.458905 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn8f8\" (UniqueName: \"kubernetes.io/projected/3b1da88a-a253-4948-85a7-20e4fb21b8d7-kube-api-access-jn8f8\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:45.458907 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.458915 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-data\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:45.459132 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.458924 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-envoy\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:45.459132 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.458932 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b1da88a-a253-4948-85a7-20e4fb21b8d7-istio-podinfo\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:45.459132 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.458940 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b1da88a-a253-4948-85a7-20e4fb21b8d7-workload-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:07:45.921584 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.921554 2564 generic.go:356] "Generic (PLEG): container finished" podID="3b1da88a-a253-4948-85a7-20e4fb21b8d7" containerID="f89bc3e55b2543c97761b8ada2e73604e32d704666ae7eb3e57eb47be18b0027" exitCode=0 Jul 28 20:07:45.921787 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.921607 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" Jul 28 20:07:45.921787 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.921654 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" event={"ID":"3b1da88a-a253-4948-85a7-20e4fb21b8d7","Type":"ContainerDied","Data":"f89bc3e55b2543c97761b8ada2e73604e32d704666ae7eb3e57eb47be18b0027"} Jul 28 20:07:45.921787 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.921705 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b" event={"ID":"3b1da88a-a253-4948-85a7-20e4fb21b8d7","Type":"ContainerDied","Data":"58eec1d307a4ef50508076cd03b58e2cf1314f590c038af00e75e1c0af61b508"} Jul 28 20:07:45.921787 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.921733 2564 scope.go:117] "RemoveContainer" containerID="f89bc3e55b2543c97761b8ada2e73604e32d704666ae7eb3e57eb47be18b0027" Jul 28 20:07:45.930564 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.930544 2564 scope.go:117] "RemoveContainer" containerID="fa07b2bf5114b5bb6232f1b49cc8f0fd569a8d8a55b93867aeab423681401729" Jul 28 20:07:45.938760 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.938745 2564 scope.go:117] "RemoveContainer" containerID="f89bc3e55b2543c97761b8ada2e73604e32d704666ae7eb3e57eb47be18b0027" Jul 28 20:07:45.938987 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:07:45.938969 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f89bc3e55b2543c97761b8ada2e73604e32d704666ae7eb3e57eb47be18b0027\": container with ID starting with f89bc3e55b2543c97761b8ada2e73604e32d704666ae7eb3e57eb47be18b0027 not found: ID does not exist" containerID="f89bc3e55b2543c97761b8ada2e73604e32d704666ae7eb3e57eb47be18b0027" Jul 28 20:07:45.939029 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.938994 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f89bc3e55b2543c97761b8ada2e73604e32d704666ae7eb3e57eb47be18b0027"} err="failed to get container status \"f89bc3e55b2543c97761b8ada2e73604e32d704666ae7eb3e57eb47be18b0027\": rpc error: code = NotFound desc = could not find container \"f89bc3e55b2543c97761b8ada2e73604e32d704666ae7eb3e57eb47be18b0027\": container with ID starting with f89bc3e55b2543c97761b8ada2e73604e32d704666ae7eb3e57eb47be18b0027 not found: ID does not exist" Jul 28 20:07:45.939029 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.939014 2564 scope.go:117] "RemoveContainer" containerID="fa07b2bf5114b5bb6232f1b49cc8f0fd569a8d8a55b93867aeab423681401729" Jul 28 20:07:45.939220 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:07:45.939205 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa07b2bf5114b5bb6232f1b49cc8f0fd569a8d8a55b93867aeab423681401729\": container with ID starting with fa07b2bf5114b5bb6232f1b49cc8f0fd569a8d8a55b93867aeab423681401729 not found: ID does not exist" containerID="fa07b2bf5114b5bb6232f1b49cc8f0fd569a8d8a55b93867aeab423681401729" Jul 28 20:07:45.939260 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.939223 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa07b2bf5114b5bb6232f1b49cc8f0fd569a8d8a55b93867aeab423681401729"} err="failed to get container status \"fa07b2bf5114b5bb6232f1b49cc8f0fd569a8d8a55b93867aeab423681401729\": rpc error: code = NotFound desc = could not find container \"fa07b2bf5114b5bb6232f1b49cc8f0fd569a8d8a55b93867aeab423681401729\": container with ID starting with fa07b2bf5114b5bb6232f1b49cc8f0fd569a8d8a55b93867aeab423681401729 not found: ID does not exist" Jul 28 20:07:45.944733 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.944689 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b"] Jul 28 20:07:45.947386 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:45.947364 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-19b71b42-openshift-default-77f5557984-jgq4b"] Jul 28 20:07:46.923792 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:07:46.923758 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b1da88a-a253-4948-85a7-20e4fb21b8d7" path="/var/lib/kubelet/pods/3b1da88a-a253-4948-85a7-20e4fb21b8d7/volumes" Jul 28 20:09:27.600361 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.600323 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r"] Jul 28 20:09:27.600849 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.600756 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67e388f5-a6b6-4824-8095-46b72ea749b2" containerName="payload-pre-processing" Jul 28 20:09:27.600849 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.600770 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="67e388f5-a6b6-4824-8095-46b72ea749b2" containerName="payload-pre-processing" Jul 28 20:09:27.600849 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.600780 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b1da88a-a253-4948-85a7-20e4fb21b8d7" containerName="istio-proxy" Jul 28 20:09:27.600849 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.600785 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b1da88a-a253-4948-85a7-20e4fb21b8d7" containerName="istio-proxy" Jul 28 20:09:27.600849 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.600804 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerName="istio-proxy" Jul 28 20:09:27.600849 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.600809 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerName="istio-proxy" Jul 28 20:09:27.600849 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.600826 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerName="istio-proxy" Jul 28 20:09:27.600849 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.600831 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerName="istio-proxy" Jul 28 20:09:27.601206 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.600884 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b1da88a-a253-4948-85a7-20e4fb21b8d7" containerName="istio-proxy" Jul 28 20:09:27.601206 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.600892 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerName="istio-proxy" Jul 28 20:09:27.601206 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.600898 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="67e388f5-a6b6-4824-8095-46b72ea749b2" containerName="payload-pre-processing" Jul 28 20:09:27.601206 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.600905 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb32653e-8048-444c-8d3d-ff3a08a177ff" containerName="istio-proxy" Jul 28 20:09:27.602978 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.602962 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.605808 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.605770 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9696d332-openshift-default-dockercfg-zj4gl\"" Jul 28 20:09:27.628341 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.628311 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r"] Jul 28 20:09:27.706433 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.706399 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-istio-data\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.706617 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.706440 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzxpl\" (UniqueName: \"kubernetes.io/projected/7314efaf-f866-4303-800f-5c7db093ca62-kube-api-access-lzxpl\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.706617 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.706477 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7314efaf-f866-4303-800f-5c7db093ca62-istiod-ca-cert\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.706617 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.706506 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-credential-socket\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.706617 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.706537 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7314efaf-f866-4303-800f-5c7db093ca62-istio-token\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.706854 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.706678 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7314efaf-f866-4303-800f-5c7db093ca62-istio-podinfo\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.706854 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.706715 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-istio-envoy\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.706854 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.706782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-workload-socket\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.706854 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.706842 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-workload-certs\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.808309 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.808273 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-istio-data\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.808309 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.808313 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzxpl\" (UniqueName: \"kubernetes.io/projected/7314efaf-f866-4303-800f-5c7db093ca62-kube-api-access-lzxpl\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.808503 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.808361 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7314efaf-f866-4303-800f-5c7db093ca62-istiod-ca-cert\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.808503 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.808401 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-credential-socket\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.808503 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.808448 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7314efaf-f866-4303-800f-5c7db093ca62-istio-token\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.808503 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.808485 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7314efaf-f866-4303-800f-5c7db093ca62-istio-podinfo\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.808752 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.808509 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-istio-envoy\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.808752 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.808554 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-workload-socket\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.808752 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.808591 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-workload-certs\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.808912 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.808763 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-istio-data\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.808912 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.808826 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-credential-socket\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.809029 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.808960 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-workload-certs\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.809157 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.809134 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-workload-socket\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.809219 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.809134 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7314efaf-f866-4303-800f-5c7db093ca62-istiod-ca-cert\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.810891 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.810871 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-istio-envoy\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.811018 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.810999 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7314efaf-f866-4303-800f-5c7db093ca62-istio-podinfo\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.817331 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.817308 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7314efaf-f866-4303-800f-5c7db093ca62-istio-token\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.817497 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.817478 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzxpl\" (UniqueName: \"kubernetes.io/projected/7314efaf-f866-4303-800f-5c7db093ca62-kube-api-access-lzxpl\") pod \"e2e-mt-9696d332-openshift-default-597698f76d-b797r\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:27.912886 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:27.912807 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:28.045779 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:28.045751 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r"] Jul 28 20:09:28.047812 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:09:28.047784 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7314efaf_f866_4303_800f_5c7db093ca62.slice/crio-37ab4b7dfe0b52340df1bbf43415a4f1ca88bf84a52580e17d4a6dedf47d8b51 WatchSource:0}: Error finding container 37ab4b7dfe0b52340df1bbf43415a4f1ca88bf84a52580e17d4a6dedf47d8b51: Status 404 returned error can't find the container with id 37ab4b7dfe0b52340df1bbf43415a4f1ca88bf84a52580e17d4a6dedf47d8b51 Jul 28 20:09:28.050088 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:28.050057 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:09:28.050183 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:28.050119 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:09:28.050183 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:28.050159 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:09:28.418577 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:28.418540 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" event={"ID":"7314efaf-f866-4303-800f-5c7db093ca62","Type":"ContainerStarted","Data":"d5920b5b382f7909ca739d2d168db369ebd9dd8c92337eb7f11de2c2f85046d2"} Jul 28 20:09:28.418780 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:28.418596 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" event={"ID":"7314efaf-f866-4303-800f-5c7db093ca62","Type":"ContainerStarted","Data":"37ab4b7dfe0b52340df1bbf43415a4f1ca88bf84a52580e17d4a6dedf47d8b51"} Jul 28 20:09:28.441821 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:28.441765 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" podStartSLOduration=1.4417488729999999 podStartE2EDuration="1.441748873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:28.44037728 +0000 UTC m=+1964.147589172" watchObservedRunningTime="2026-07-28 20:09:28.441748873 +0000 UTC m=+1964.148960819" Jul 28 20:09:28.918830 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:28.918803 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:29.917988 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:29.917960 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:30.429449 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:30.429421 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:30.430375 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:30.430354 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:31.556933 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.556900 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5"] Jul 28 20:09:31.557375 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.557358 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b1da88a-a253-4948-85a7-20e4fb21b8d7" containerName="istio-proxy" Jul 28 20:09:31.557426 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.557377 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b1da88a-a253-4948-85a7-20e4fb21b8d7" containerName="istio-proxy" Jul 28 20:09:31.557461 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.557427 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b1da88a-a253-4948-85a7-20e4fb21b8d7" containerName="istio-proxy" Jul 28 20:09:31.559433 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.559417 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" Jul 28 20:09:31.566654 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.562407 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-9696d332\"" Jul 28 20:09:31.566654 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.562886 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-9696d332-dockercfg-rnzrg\"" Jul 28 20:09:31.573933 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.573907 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5"] Jul 28 20:09:31.641675 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.641615 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11e0f194-6cf8-4055-8045-4fa9871dc358-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5\" (UID: \"11e0f194-6cf8-4055-8045-4fa9871dc358\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" Jul 28 20:09:31.641871 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.641760 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46dd6\" (UniqueName: \"kubernetes.io/projected/11e0f194-6cf8-4055-8045-4fa9871dc358-kube-api-access-46dd6\") pod \"payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5\" (UID: \"11e0f194-6cf8-4055-8045-4fa9871dc358\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" Jul 28 20:09:31.645301 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.645275 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r"] Jul 28 20:09:31.647676 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.647660 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" Jul 28 20:09:31.661226 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.661199 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r"] Jul 28 20:09:31.742931 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.742900 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lsk2\" (UniqueName: \"kubernetes.io/projected/11d56eb6-8b4a-42d2-8686-2f264bef658e-kube-api-access-2lsk2\") pod \"payload-processing-e2e-mt-9696d332-6646df7546-hkw4r\" (UID: \"11d56eb6-8b4a-42d2-8686-2f264bef658e\") " pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" Jul 28 20:09:31.743129 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.742955 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11e0f194-6cf8-4055-8045-4fa9871dc358-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5\" (UID: \"11e0f194-6cf8-4055-8045-4fa9871dc358\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" Jul 28 20:09:31.743129 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.743046 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11d56eb6-8b4a-42d2-8686-2f264bef658e-plugins-config-volume\") pod \"payload-processing-e2e-mt-9696d332-6646df7546-hkw4r\" (UID: \"11d56eb6-8b4a-42d2-8686-2f264bef658e\") " pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" Jul 28 20:09:31.743129 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.743095 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46dd6\" (UniqueName: \"kubernetes.io/projected/11e0f194-6cf8-4055-8045-4fa9871dc358-kube-api-access-46dd6\") pod \"payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5\" (UID: \"11e0f194-6cf8-4055-8045-4fa9871dc358\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" Jul 28 20:09:31.743571 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.743552 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11e0f194-6cf8-4055-8045-4fa9871dc358-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5\" (UID: \"11e0f194-6cf8-4055-8045-4fa9871dc358\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" Jul 28 20:09:31.751432 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.751405 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46dd6\" (UniqueName: \"kubernetes.io/projected/11e0f194-6cf8-4055-8045-4fa9871dc358-kube-api-access-46dd6\") pod \"payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5\" (UID: \"11e0f194-6cf8-4055-8045-4fa9871dc358\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" Jul 28 20:09:31.843964 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.843865 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lsk2\" (UniqueName: \"kubernetes.io/projected/11d56eb6-8b4a-42d2-8686-2f264bef658e-kube-api-access-2lsk2\") pod \"payload-processing-e2e-mt-9696d332-6646df7546-hkw4r\" (UID: \"11d56eb6-8b4a-42d2-8686-2f264bef658e\") " pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" Jul 28 20:09:31.844138 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.843985 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11d56eb6-8b4a-42d2-8686-2f264bef658e-plugins-config-volume\") pod \"payload-processing-e2e-mt-9696d332-6646df7546-hkw4r\" (UID: \"11d56eb6-8b4a-42d2-8686-2f264bef658e\") " pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" Jul 28 20:09:31.844674 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.844624 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11d56eb6-8b4a-42d2-8686-2f264bef658e-plugins-config-volume\") pod \"payload-processing-e2e-mt-9696d332-6646df7546-hkw4r\" (UID: \"11d56eb6-8b4a-42d2-8686-2f264bef658e\") " pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" Jul 28 20:09:31.852295 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.852274 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lsk2\" (UniqueName: \"kubernetes.io/projected/11d56eb6-8b4a-42d2-8686-2f264bef658e-kube-api-access-2lsk2\") pod \"payload-processing-e2e-mt-9696d332-6646df7546-hkw4r\" (UID: \"11d56eb6-8b4a-42d2-8686-2f264bef658e\") " pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" Jul 28 20:09:31.877377 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.877352 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" Jul 28 20:09:31.956966 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:31.956936 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" Jul 28 20:09:32.016745 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:32.016700 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5"] Jul 28 20:09:32.019030 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:09:32.018982 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11e0f194_6cf8_4055_8045_4fa9871dc358.slice/crio-9b5a64c489f112b7bc599b837c06105e957fc27c6dc0601bd5b0aa8bc386fad5 WatchSource:0}: Error finding container 9b5a64c489f112b7bc599b837c06105e957fc27c6dc0601bd5b0aa8bc386fad5: Status 404 returned error can't find the container with id 9b5a64c489f112b7bc599b837c06105e957fc27c6dc0601bd5b0aa8bc386fad5 Jul 28 20:09:32.106157 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:32.106133 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r"] Jul 28 20:09:32.108131 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:09:32.108107 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d56eb6_8b4a_42d2_8686_2f264bef658e.slice/crio-84a72246ee3e96a6653db5025771769ecc0a3cfaad9caa89e6f21d99457c8f86 WatchSource:0}: Error finding container 84a72246ee3e96a6653db5025771769ecc0a3cfaad9caa89e6f21d99457c8f86: Status 404 returned error can't find the container with id 84a72246ee3e96a6653db5025771769ecc0a3cfaad9caa89e6f21d99457c8f86 Jul 28 20:09:32.440580 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:32.440479 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" event={"ID":"11d56eb6-8b4a-42d2-8686-2f264bef658e","Type":"ContainerStarted","Data":"0259372b518233bbe7314756278610215e8434fcd04fa5c8531abf48837b6b06"} Jul 28 20:09:32.440580 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:32.440551 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" event={"ID":"11d56eb6-8b4a-42d2-8686-2f264bef658e","Type":"ContainerStarted","Data":"84a72246ee3e96a6653db5025771769ecc0a3cfaad9caa89e6f21d99457c8f86"} Jul 28 20:09:32.440855 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:32.440597 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" Jul 28 20:09:32.442258 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:32.442227 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" event={"ID":"11e0f194-6cf8-4055-8045-4fa9871dc358","Type":"ContainerStarted","Data":"9ebe2cf41806e1afe7e1f6f1ea131a9947a0fee8c25fa5420e92501620e0a60b"} Jul 28 20:09:32.442397 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:32.442283 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" event={"ID":"11e0f194-6cf8-4055-8045-4fa9871dc358","Type":"ContainerStarted","Data":"9b5a64c489f112b7bc599b837c06105e957fc27c6dc0601bd5b0aa8bc386fad5"} Jul 28 20:09:32.457786 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:32.457737 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" podStartSLOduration=1.457717845 podStartE2EDuration="1.457717845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:32.455875506 +0000 UTC m=+1968.163087400" watchObservedRunningTime="2026-07-28 20:09:32.457717845 +0000 UTC m=+1968.164929721" Jul 28 20:09:32.472351 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:32.472305 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" podStartSLOduration=1.472290896 podStartE2EDuration="1.472290896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:32.470896217 +0000 UTC m=+1968.178108136" watchObservedRunningTime="2026-07-28 20:09:32.472290896 +0000 UTC m=+1968.179502788" Jul 28 20:09:33.447280 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:33.447252 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" Jul 28 20:09:43.449679 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:43.449646 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" Jul 28 20:09:44.454612 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:44.454520 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" Jul 28 20:09:44.506616 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:44.506584 2564 generic.go:356] "Generic (PLEG): container finished" podID="7314efaf-f866-4303-800f-5c7db093ca62" containerID="d5920b5b382f7909ca739d2d168db369ebd9dd8c92337eb7f11de2c2f85046d2" exitCode=0 Jul 28 20:09:44.506783 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:44.506659 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" event={"ID":"7314efaf-f866-4303-800f-5c7db093ca62","Type":"ContainerDied","Data":"d5920b5b382f7909ca739d2d168db369ebd9dd8c92337eb7f11de2c2f85046d2"} Jul 28 20:09:44.507098 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:44.507082 2564 scope.go:117] "RemoveContainer" containerID="d5920b5b382f7909ca739d2d168db369ebd9dd8c92337eb7f11de2c2f85046d2" Jul 28 20:09:44.508203 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:44.508175 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:09:44.508274 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:44.508241 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:09:44.508274 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:44.508269 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:09:45.429459 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:45.429424 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:45.513540 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:45.513509 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" event={"ID":"7314efaf-f866-4303-800f-5c7db093ca62","Type":"ContainerStarted","Data":"c521d3bcab3f2de3d35ff3dbefcaa189ccab2ab7028c245d06f52d43c503d426"} Jul 28 20:09:45.913536 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:45.913499 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:45.914682 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:45.914657 2564 patch_prober.go:28] interesting pod/e2e-mt-9696d332-openshift-default-597698f76d-b797r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 28 20:09:45.914817 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:45.914702 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" podUID="7314efaf-f866-4303-800f-5c7db093ca62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 28 20:09:46.913529 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:46.913490 2564 patch_prober.go:28] interesting pod/e2e-mt-9696d332-openshift-default-597698f76d-b797r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 28 20:09:46.913966 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:46.913554 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" podUID="7314efaf-f866-4303-800f-5c7db093ca62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 28 20:09:47.914115 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:47.914077 2564 patch_prober.go:28] interesting pod/e2e-mt-9696d332-openshift-default-597698f76d-b797r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 28 20:09:47.914511 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:47.914135 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" podUID="7314efaf-f866-4303-800f-5c7db093ca62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 28 20:09:48.913147 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:48.913113 2564 patch_prober.go:28] interesting pod/e2e-mt-9696d332-openshift-default-597698f76d-b797r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 28 20:09:48.913350 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:48.913172 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" podUID="7314efaf-f866-4303-800f-5c7db093ca62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 28 20:09:49.917482 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:49.917451 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:49.917985 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:49.917830 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:09:49.918346 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:09:49.918328 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:10:18.181290 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.181258 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r"] Jul 28 20:10:18.181874 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.181508 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" podUID="11d56eb6-8b4a-42d2-8686-2f264bef658e" containerName="payload-processing" containerID="cri-o://0259372b518233bbe7314756278610215e8434fcd04fa5c8531abf48837b6b06" gracePeriod=30 Jul 28 20:10:18.186675 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.186647 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5"] Jul 28 20:10:18.186864 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.186826 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" podUID="11e0f194-6cf8-4055-8045-4fa9871dc358" containerName="payload-pre-processing" containerID="cri-o://9ebe2cf41806e1afe7e1f6f1ea131a9947a0fee8c25fa5420e92501620e0a60b" gracePeriod=30 Jul 28 20:10:18.460697 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.460674 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" Jul 28 20:10:18.463764 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.463743 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" Jul 28 20:10:18.606489 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.606455 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lsk2\" (UniqueName: \"kubernetes.io/projected/11d56eb6-8b4a-42d2-8686-2f264bef658e-kube-api-access-2lsk2\") pod \"11d56eb6-8b4a-42d2-8686-2f264bef658e\" (UID: \"11d56eb6-8b4a-42d2-8686-2f264bef658e\") " Jul 28 20:10:18.606717 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.606519 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11d56eb6-8b4a-42d2-8686-2f264bef658e-plugins-config-volume\") pod \"11d56eb6-8b4a-42d2-8686-2f264bef658e\" (UID: \"11d56eb6-8b4a-42d2-8686-2f264bef658e\") " Jul 28 20:10:18.606717 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.606559 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11e0f194-6cf8-4055-8045-4fa9871dc358-plugins-config-volume\") pod \"11e0f194-6cf8-4055-8045-4fa9871dc358\" (UID: \"11e0f194-6cf8-4055-8045-4fa9871dc358\") " Jul 28 20:10:18.606717 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.606587 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46dd6\" (UniqueName: \"kubernetes.io/projected/11e0f194-6cf8-4055-8045-4fa9871dc358-kube-api-access-46dd6\") pod \"11e0f194-6cf8-4055-8045-4fa9871dc358\" (UID: \"11e0f194-6cf8-4055-8045-4fa9871dc358\") " Jul 28 20:10:18.606985 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.606959 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11e0f194-6cf8-4055-8045-4fa9871dc358-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "11e0f194-6cf8-4055-8045-4fa9871dc358" (UID: "11e0f194-6cf8-4055-8045-4fa9871dc358"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:10:18.607039 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.606959 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11d56eb6-8b4a-42d2-8686-2f264bef658e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "11d56eb6-8b4a-42d2-8686-2f264bef658e" (UID: "11d56eb6-8b4a-42d2-8686-2f264bef658e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:10:18.608773 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.608746 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11e0f194-6cf8-4055-8045-4fa9871dc358-kube-api-access-46dd6" (OuterVolumeSpecName: "kube-api-access-46dd6") pod "11e0f194-6cf8-4055-8045-4fa9871dc358" (UID: "11e0f194-6cf8-4055-8045-4fa9871dc358"). InnerVolumeSpecName "kube-api-access-46dd6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:10:18.608844 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.608784 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11d56eb6-8b4a-42d2-8686-2f264bef658e-kube-api-access-2lsk2" (OuterVolumeSpecName: "kube-api-access-2lsk2") pod "11d56eb6-8b4a-42d2-8686-2f264bef658e" (UID: "11d56eb6-8b4a-42d2-8686-2f264bef658e"). InnerVolumeSpecName "kube-api-access-2lsk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:10:18.678770 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.678737 2564 generic.go:356] "Generic (PLEG): container finished" podID="11e0f194-6cf8-4055-8045-4fa9871dc358" containerID="9ebe2cf41806e1afe7e1f6f1ea131a9947a0fee8c25fa5420e92501620e0a60b" exitCode=0 Jul 28 20:10:18.678899 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.678785 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" Jul 28 20:10:18.678899 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.678828 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" event={"ID":"11e0f194-6cf8-4055-8045-4fa9871dc358","Type":"ContainerDied","Data":"9ebe2cf41806e1afe7e1f6f1ea131a9947a0fee8c25fa5420e92501620e0a60b"} Jul 28 20:10:18.678899 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.678882 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5" event={"ID":"11e0f194-6cf8-4055-8045-4fa9871dc358","Type":"ContainerDied","Data":"9b5a64c489f112b7bc599b837c06105e957fc27c6dc0601bd5b0aa8bc386fad5"} Jul 28 20:10:18.679056 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.678913 2564 scope.go:117] "RemoveContainer" containerID="9ebe2cf41806e1afe7e1f6f1ea131a9947a0fee8c25fa5420e92501620e0a60b" Jul 28 20:10:18.680189 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.680169 2564 generic.go:356] "Generic (PLEG): container finished" podID="11d56eb6-8b4a-42d2-8686-2f264bef658e" containerID="0259372b518233bbe7314756278610215e8434fcd04fa5c8531abf48837b6b06" exitCode=0 Jul 28 20:10:18.680276 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.680199 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" event={"ID":"11d56eb6-8b4a-42d2-8686-2f264bef658e","Type":"ContainerDied","Data":"0259372b518233bbe7314756278610215e8434fcd04fa5c8531abf48837b6b06"} Jul 28 20:10:18.680276 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.680232 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" Jul 28 20:10:18.680276 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.680245 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r" event={"ID":"11d56eb6-8b4a-42d2-8686-2f264bef658e","Type":"ContainerDied","Data":"84a72246ee3e96a6653db5025771769ecc0a3cfaad9caa89e6f21d99457c8f86"} Jul 28 20:10:18.689137 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.689121 2564 scope.go:117] "RemoveContainer" containerID="9ebe2cf41806e1afe7e1f6f1ea131a9947a0fee8c25fa5420e92501620e0a60b" Jul 28 20:10:18.689437 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:10:18.689413 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ebe2cf41806e1afe7e1f6f1ea131a9947a0fee8c25fa5420e92501620e0a60b\": container with ID starting with 9ebe2cf41806e1afe7e1f6f1ea131a9947a0fee8c25fa5420e92501620e0a60b not found: ID does not exist" containerID="9ebe2cf41806e1afe7e1f6f1ea131a9947a0fee8c25fa5420e92501620e0a60b" Jul 28 20:10:18.689539 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.689445 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ebe2cf41806e1afe7e1f6f1ea131a9947a0fee8c25fa5420e92501620e0a60b"} err="failed to get container status \"9ebe2cf41806e1afe7e1f6f1ea131a9947a0fee8c25fa5420e92501620e0a60b\": rpc error: code = NotFound desc = could not find container \"9ebe2cf41806e1afe7e1f6f1ea131a9947a0fee8c25fa5420e92501620e0a60b\": container with ID starting with 9ebe2cf41806e1afe7e1f6f1ea131a9947a0fee8c25fa5420e92501620e0a60b not found: ID does not exist" Jul 28 20:10:18.689539 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.689465 2564 scope.go:117] "RemoveContainer" containerID="0259372b518233bbe7314756278610215e8434fcd04fa5c8531abf48837b6b06" Jul 28 20:10:18.697006 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.696990 2564 scope.go:117] "RemoveContainer" containerID="0259372b518233bbe7314756278610215e8434fcd04fa5c8531abf48837b6b06" Jul 28 20:10:18.697233 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:10:18.697216 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0259372b518233bbe7314756278610215e8434fcd04fa5c8531abf48837b6b06\": container with ID starting with 0259372b518233bbe7314756278610215e8434fcd04fa5c8531abf48837b6b06 not found: ID does not exist" containerID="0259372b518233bbe7314756278610215e8434fcd04fa5c8531abf48837b6b06" Jul 28 20:10:18.697280 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.697238 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0259372b518233bbe7314756278610215e8434fcd04fa5c8531abf48837b6b06"} err="failed to get container status \"0259372b518233bbe7314756278610215e8434fcd04fa5c8531abf48837b6b06\": rpc error: code = NotFound desc = could not find container \"0259372b518233bbe7314756278610215e8434fcd04fa5c8531abf48837b6b06\": container with ID starting with 0259372b518233bbe7314756278610215e8434fcd04fa5c8531abf48837b6b06 not found: ID does not exist" Jul 28 20:10:18.701205 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.701185 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5"] Jul 28 20:10:18.705419 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.705399 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9696d332-75487db7b8-shjj5"] Jul 28 20:10:18.707309 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.707290 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46dd6\" (UniqueName: \"kubernetes.io/projected/11e0f194-6cf8-4055-8045-4fa9871dc358-kube-api-access-46dd6\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:10:18.707309 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.707309 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lsk2\" (UniqueName: \"kubernetes.io/projected/11d56eb6-8b4a-42d2-8686-2f264bef658e-kube-api-access-2lsk2\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:10:18.707445 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.707319 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11d56eb6-8b4a-42d2-8686-2f264bef658e-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:10:18.707445 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.707329 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11e0f194-6cf8-4055-8045-4fa9871dc358-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:10:18.717951 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.717931 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r"] Jul 28 20:10:18.719506 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.719488 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9696d332-6646df7546-hkw4r"] Jul 28 20:10:18.921699 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.921662 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11d56eb6-8b4a-42d2-8686-2f264bef658e" path="/var/lib/kubelet/pods/11d56eb6-8b4a-42d2-8686-2f264bef658e/volumes" Jul 28 20:10:18.922080 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:18.922065 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11e0f194-6cf8-4055-8045-4fa9871dc358" path="/var/lib/kubelet/pods/11e0f194-6cf8-4055-8045-4fa9871dc358/volumes" Jul 28 20:10:24.729244 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:24.729213 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r"] Jul 28 20:10:24.729854 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:24.729406 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" podUID="7314efaf-f866-4303-800f-5c7db093ca62" containerName="istio-proxy" containerID="cri-o://c521d3bcab3f2de3d35ff3dbefcaa189ccab2ab7028c245d06f52d43c503d426" gracePeriod=30 Jul 28 20:10:29.980150 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:29.980124 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:10:30.117320 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.117291 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7314efaf-f866-4303-800f-5c7db093ca62-istio-token\") pod \"7314efaf-f866-4303-800f-5c7db093ca62\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " Jul 28 20:10:30.117505 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.117341 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7314efaf-f866-4303-800f-5c7db093ca62-istio-podinfo\") pod \"7314efaf-f866-4303-800f-5c7db093ca62\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " Jul 28 20:10:30.117505 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.117367 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-credential-socket\") pod \"7314efaf-f866-4303-800f-5c7db093ca62\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " Jul 28 20:10:30.117648 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.117515 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-istio-data\") pod \"7314efaf-f866-4303-800f-5c7db093ca62\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " Jul 28 20:10:30.117648 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.117563 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7314efaf-f866-4303-800f-5c7db093ca62-istiod-ca-cert\") pod \"7314efaf-f866-4303-800f-5c7db093ca62\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " Jul 28 20:10:30.117648 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.117594 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7314efaf-f866-4303-800f-5c7db093ca62" (UID: "7314efaf-f866-4303-800f-5c7db093ca62"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:10:30.117822 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.117650 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-workload-certs\") pod \"7314efaf-f866-4303-800f-5c7db093ca62\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " Jul 28 20:10:30.117822 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.117686 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzxpl\" (UniqueName: \"kubernetes.io/projected/7314efaf-f866-4303-800f-5c7db093ca62-kube-api-access-lzxpl\") pod \"7314efaf-f866-4303-800f-5c7db093ca62\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " Jul 28 20:10:30.117822 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.117751 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-workload-socket\") pod \"7314efaf-f866-4303-800f-5c7db093ca62\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " Jul 28 20:10:30.117822 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.117782 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-istio-envoy\") pod \"7314efaf-f866-4303-800f-5c7db093ca62\" (UID: \"7314efaf-f866-4303-800f-5c7db093ca62\") " Jul 28 20:10:30.118016 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.117915 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7314efaf-f866-4303-800f-5c7db093ca62" (UID: "7314efaf-f866-4303-800f-5c7db093ca62"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:10:30.118016 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.117954 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7314efaf-f866-4303-800f-5c7db093ca62-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7314efaf-f866-4303-800f-5c7db093ca62" (UID: "7314efaf-f866-4303-800f-5c7db093ca62"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:10:30.118393 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.118176 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-credential-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:10:30.118393 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.118198 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7314efaf-f866-4303-800f-5c7db093ca62-istiod-ca-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:10:30.118393 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.118213 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-workload-certs\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:10:30.118393 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.118357 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7314efaf-f866-4303-800f-5c7db093ca62" (UID: "7314efaf-f866-4303-800f-5c7db093ca62"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:10:30.118697 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.118383 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-istio-data" (OuterVolumeSpecName: "istio-data") pod "7314efaf-f866-4303-800f-5c7db093ca62" (UID: "7314efaf-f866-4303-800f-5c7db093ca62"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:10:30.119771 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.119743 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7314efaf-f866-4303-800f-5c7db093ca62-istio-token" (OuterVolumeSpecName: "istio-token") pod "7314efaf-f866-4303-800f-5c7db093ca62" (UID: "7314efaf-f866-4303-800f-5c7db093ca62"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:10:30.120169 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.120128 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7314efaf-f866-4303-800f-5c7db093ca62-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7314efaf-f866-4303-800f-5c7db093ca62" (UID: "7314efaf-f866-4303-800f-5c7db093ca62"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 20:10:30.120244 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.120146 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7314efaf-f866-4303-800f-5c7db093ca62-kube-api-access-lzxpl" (OuterVolumeSpecName: "kube-api-access-lzxpl") pod "7314efaf-f866-4303-800f-5c7db093ca62" (UID: "7314efaf-f866-4303-800f-5c7db093ca62"). InnerVolumeSpecName "kube-api-access-lzxpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:10:30.120244 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.120168 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7314efaf-f866-4303-800f-5c7db093ca62" (UID: "7314efaf-f866-4303-800f-5c7db093ca62"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:10:30.219099 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.219071 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7314efaf-f866-4303-800f-5c7db093ca62-istio-token\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:10:30.219099 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.219096 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7314efaf-f866-4303-800f-5c7db093ca62-istio-podinfo\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:10:30.219099 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.219105 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-istio-data\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:10:30.219313 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.219114 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzxpl\" (UniqueName: \"kubernetes.io/projected/7314efaf-f866-4303-800f-5c7db093ca62-kube-api-access-lzxpl\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:10:30.219313 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.219123 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-workload-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:10:30.219313 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.219132 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7314efaf-f866-4303-800f-5c7db093ca62-istio-envoy\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:10:30.748696 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.748658 2564 generic.go:356] "Generic (PLEG): container finished" podID="7314efaf-f866-4303-800f-5c7db093ca62" containerID="c521d3bcab3f2de3d35ff3dbefcaa189ccab2ab7028c245d06f52d43c503d426" exitCode=0 Jul 28 20:10:30.748876 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.748706 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" Jul 28 20:10:30.748876 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.748743 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" event={"ID":"7314efaf-f866-4303-800f-5c7db093ca62","Type":"ContainerDied","Data":"c521d3bcab3f2de3d35ff3dbefcaa189ccab2ab7028c245d06f52d43c503d426"} Jul 28 20:10:30.748876 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.748819 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r" event={"ID":"7314efaf-f866-4303-800f-5c7db093ca62","Type":"ContainerDied","Data":"37ab4b7dfe0b52340df1bbf43415a4f1ca88bf84a52580e17d4a6dedf47d8b51"} Jul 28 20:10:30.748876 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.748842 2564 scope.go:117] "RemoveContainer" containerID="c521d3bcab3f2de3d35ff3dbefcaa189ccab2ab7028c245d06f52d43c503d426" Jul 28 20:10:30.758364 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.758349 2564 scope.go:117] "RemoveContainer" containerID="d5920b5b382f7909ca739d2d168db369ebd9dd8c92337eb7f11de2c2f85046d2" Jul 28 20:10:30.765764 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.765738 2564 scope.go:117] "RemoveContainer" containerID="c521d3bcab3f2de3d35ff3dbefcaa189ccab2ab7028c245d06f52d43c503d426" Jul 28 20:10:30.765996 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:10:30.765977 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c521d3bcab3f2de3d35ff3dbefcaa189ccab2ab7028c245d06f52d43c503d426\": container with ID starting with c521d3bcab3f2de3d35ff3dbefcaa189ccab2ab7028c245d06f52d43c503d426 not found: ID does not exist" containerID="c521d3bcab3f2de3d35ff3dbefcaa189ccab2ab7028c245d06f52d43c503d426" Jul 28 20:10:30.766061 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.766003 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c521d3bcab3f2de3d35ff3dbefcaa189ccab2ab7028c245d06f52d43c503d426"} err="failed to get container status \"c521d3bcab3f2de3d35ff3dbefcaa189ccab2ab7028c245d06f52d43c503d426\": rpc error: code = NotFound desc = could not find container \"c521d3bcab3f2de3d35ff3dbefcaa189ccab2ab7028c245d06f52d43c503d426\": container with ID starting with c521d3bcab3f2de3d35ff3dbefcaa189ccab2ab7028c245d06f52d43c503d426 not found: ID does not exist" Jul 28 20:10:30.766061 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.766019 2564 scope.go:117] "RemoveContainer" containerID="d5920b5b382f7909ca739d2d168db369ebd9dd8c92337eb7f11de2c2f85046d2" Jul 28 20:10:30.766263 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:10:30.766245 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5920b5b382f7909ca739d2d168db369ebd9dd8c92337eb7f11de2c2f85046d2\": container with ID starting with d5920b5b382f7909ca739d2d168db369ebd9dd8c92337eb7f11de2c2f85046d2 not found: ID does not exist" containerID="d5920b5b382f7909ca739d2d168db369ebd9dd8c92337eb7f11de2c2f85046d2" Jul 28 20:10:30.766303 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.766269 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5920b5b382f7909ca739d2d168db369ebd9dd8c92337eb7f11de2c2f85046d2"} err="failed to get container status \"d5920b5b382f7909ca739d2d168db369ebd9dd8c92337eb7f11de2c2f85046d2\": rpc error: code = NotFound desc = could not find container \"d5920b5b382f7909ca739d2d168db369ebd9dd8c92337eb7f11de2c2f85046d2\": container with ID starting with d5920b5b382f7909ca739d2d168db369ebd9dd8c92337eb7f11de2c2f85046d2 not found: ID does not exist" Jul 28 20:10:30.771362 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.771342 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r"] Jul 28 20:10:30.774599 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.774579 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9696d332-openshift-default-597698f76d-b797r"] Jul 28 20:10:30.919400 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:30.919372 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7314efaf-f866-4303-800f-5c7db093ca62" path="/var/lib/kubelet/pods/7314efaf-f866-4303-800f-5c7db093ca62/volumes" Jul 28 20:10:44.568962 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.568928 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5"] Jul 28 20:10:44.569474 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.569454 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7314efaf-f866-4303-800f-5c7db093ca62" containerName="istio-proxy" Jul 28 20:10:44.569533 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.569478 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7314efaf-f866-4303-800f-5c7db093ca62" containerName="istio-proxy" Jul 28 20:10:44.569533 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.569490 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11d56eb6-8b4a-42d2-8686-2f264bef658e" containerName="payload-processing" Jul 28 20:10:44.569533 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.569496 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="11d56eb6-8b4a-42d2-8686-2f264bef658e" containerName="payload-processing" Jul 28 20:10:44.569533 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.569513 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11e0f194-6cf8-4055-8045-4fa9871dc358" containerName="payload-pre-processing" Jul 28 20:10:44.569533 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.569518 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="11e0f194-6cf8-4055-8045-4fa9871dc358" containerName="payload-pre-processing" Jul 28 20:10:44.569726 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.569600 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7314efaf-f866-4303-800f-5c7db093ca62" containerName="istio-proxy" Jul 28 20:10:44.569726 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.569611 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="11e0f194-6cf8-4055-8045-4fa9871dc358" containerName="payload-pre-processing" Jul 28 20:10:44.569726 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.569623 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="11d56eb6-8b4a-42d2-8686-2f264bef658e" containerName="payload-processing" Jul 28 20:10:44.569726 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.569629 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7314efaf-f866-4303-800f-5c7db093ca62" containerName="istio-proxy" Jul 28 20:10:44.572625 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.572609 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.575736 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.575717 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d517388f-openshift-default-dockercfg-nt72k\"" Jul 28 20:10:44.586069 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.586045 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5"] Jul 28 20:10:44.656777 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.656737 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-workload-certs\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.656777 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.656785 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-workload-socket\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.657002 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.656809 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-istio-envoy\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.657002 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.656906 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-credential-socket\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.657002 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.656968 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-istio-data\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.657120 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.657011 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2594252-5835-4722-a7c2-0280ac188eb6-istio-podinfo\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.657120 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.657089 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2594252-5835-4722-a7c2-0280ac188eb6-istiod-ca-cert\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.657186 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.657139 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vklwn\" (UniqueName: \"kubernetes.io/projected/c2594252-5835-4722-a7c2-0280ac188eb6-kube-api-access-vklwn\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.657186 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.657169 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2594252-5835-4722-a7c2-0280ac188eb6-istio-token\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.758236 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.758200 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2594252-5835-4722-a7c2-0280ac188eb6-istiod-ca-cert\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.758418 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.758254 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vklwn\" (UniqueName: \"kubernetes.io/projected/c2594252-5835-4722-a7c2-0280ac188eb6-kube-api-access-vklwn\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.758418 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.758282 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2594252-5835-4722-a7c2-0280ac188eb6-istio-token\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.758418 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.758312 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-workload-certs\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.758418 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.758341 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-workload-socket\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.758418 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.758363 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-istio-envoy\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.758723 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.758497 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-credential-socket\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.758723 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.758547 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-istio-data\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.758723 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.758593 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2594252-5835-4722-a7c2-0280ac188eb6-istio-podinfo\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.758885 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.758771 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-workload-certs\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.758885 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.758842 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-workload-socket\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.758967 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.758941 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-istio-data\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.759010 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.758969 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2594252-5835-4722-a7c2-0280ac188eb6-istiod-ca-cert\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.759227 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.759205 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-credential-socket\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.760601 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.760584 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-istio-envoy\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.760910 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.760895 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2594252-5835-4722-a7c2-0280ac188eb6-istio-podinfo\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.766820 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.766795 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2594252-5835-4722-a7c2-0280ac188eb6-istio-token\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.767278 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.767260 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vklwn\" (UniqueName: \"kubernetes.io/projected/c2594252-5835-4722-a7c2-0280ac188eb6-kube-api-access-vklwn\") pod \"e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:44.885673 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.885574 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d517388f-openshift-default-dockercfg-nt72k\"" Jul 28 20:10:44.892789 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:44.892769 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:45.030699 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:45.030567 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5"] Jul 28 20:10:45.033863 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:10:45.033833 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2594252_5835_4722_a7c2_0280ac188eb6.slice/crio-d8225b4279746dcd86c876699579e94742ea1f579632d01e1a3d968c6c31cdfa WatchSource:0}: Error finding container d8225b4279746dcd86c876699579e94742ea1f579632d01e1a3d968c6c31cdfa: Status 404 returned error can't find the container with id d8225b4279746dcd86c876699579e94742ea1f579632d01e1a3d968c6c31cdfa Jul 28 20:10:45.036303 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:45.036268 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:10:45.036400 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:45.036356 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:10:45.036400 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:45.036386 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:10:45.822142 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:45.822106 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" event={"ID":"c2594252-5835-4722-a7c2-0280ac188eb6","Type":"ContainerStarted","Data":"330a0e80b8bbb7ce773b8639ea4fc16c363b4b880b32807cab63e728840afb99"} Jul 28 20:10:45.822532 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:45.822158 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" event={"ID":"c2594252-5835-4722-a7c2-0280ac188eb6","Type":"ContainerStarted","Data":"d8225b4279746dcd86c876699579e94742ea1f579632d01e1a3d968c6c31cdfa"} Jul 28 20:10:45.844559 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:45.844511 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" podStartSLOduration=1.84449512 podStartE2EDuration="1.84449512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:10:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:10:45.843676694 +0000 UTC m=+2041.550888791" watchObservedRunningTime="2026-07-28 20:10:45.84449512 +0000 UTC m=+2041.551707012" Jul 28 20:10:45.893685 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:45.893658 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:46.899506 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:46.899469 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:47.832087 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:47.832058 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:47.833115 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:47.833098 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:10:48.977255 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:48.977220 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q"] Jul 28 20:10:48.977820 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:48.977800 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7314efaf-f866-4303-800f-5c7db093ca62" containerName="istio-proxy" Jul 28 20:10:48.977871 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:48.977823 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7314efaf-f866-4303-800f-5c7db093ca62" containerName="istio-proxy" Jul 28 20:10:48.980312 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:48.980288 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" Jul 28 20:10:48.983198 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:48.983178 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-d517388f\"" Jul 28 20:10:48.983283 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:48.983217 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-d517388f-dockercfg-tcrvj\"" Jul 28 20:10:48.999925 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:48.999894 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q"] Jul 28 20:10:49.098649 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:49.098600 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9l76\" (UniqueName: \"kubernetes.io/projected/4ffc65e2-ed14-4d33-92e1-25fd969b10c0-kube-api-access-x9l76\") pod \"payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q\" (UID: \"4ffc65e2-ed14-4d33-92e1-25fd969b10c0\") " pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" Jul 28 20:10:49.098874 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:49.098849 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ffc65e2-ed14-4d33-92e1-25fd969b10c0-plugins-config-volume\") pod \"payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q\" (UID: \"4ffc65e2-ed14-4d33-92e1-25fd969b10c0\") " pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" Jul 28 20:10:49.200390 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:49.200354 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ffc65e2-ed14-4d33-92e1-25fd969b10c0-plugins-config-volume\") pod \"payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q\" (UID: \"4ffc65e2-ed14-4d33-92e1-25fd969b10c0\") " pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" Jul 28 20:10:49.200582 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:49.200400 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9l76\" (UniqueName: \"kubernetes.io/projected/4ffc65e2-ed14-4d33-92e1-25fd969b10c0-kube-api-access-x9l76\") pod \"payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q\" (UID: \"4ffc65e2-ed14-4d33-92e1-25fd969b10c0\") " pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" Jul 28 20:10:49.201119 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:49.201093 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ffc65e2-ed14-4d33-92e1-25fd969b10c0-plugins-config-volume\") pod \"payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q\" (UID: \"4ffc65e2-ed14-4d33-92e1-25fd969b10c0\") " pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" Jul 28 20:10:49.208572 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:49.208542 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9l76\" (UniqueName: \"kubernetes.io/projected/4ffc65e2-ed14-4d33-92e1-25fd969b10c0-kube-api-access-x9l76\") pod \"payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q\" (UID: \"4ffc65e2-ed14-4d33-92e1-25fd969b10c0\") " pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" Jul 28 20:10:49.302137 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:49.302108 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" Jul 28 20:10:49.433194 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:49.433168 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q"] Jul 28 20:10:49.435403 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:10:49.435378 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ffc65e2_ed14_4d33_92e1_25fd969b10c0.slice/crio-faf99a32f7eddf29aeb4089c03ecfeac1906342cc6d2a858809b431b486152e4 WatchSource:0}: Error finding container faf99a32f7eddf29aeb4089c03ecfeac1906342cc6d2a858809b431b486152e4: Status 404 returned error can't find the container with id faf99a32f7eddf29aeb4089c03ecfeac1906342cc6d2a858809b431b486152e4 Jul 28 20:10:49.843967 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:49.843937 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" event={"ID":"4ffc65e2-ed14-4d33-92e1-25fd969b10c0","Type":"ContainerStarted","Data":"2d2d8e6506f786f8909c248ea50cd90251ad6d117bc33e995387df0a2151efcd"} Jul 28 20:10:49.844244 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:49.843986 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" event={"ID":"4ffc65e2-ed14-4d33-92e1-25fd969b10c0","Type":"ContainerStarted","Data":"faf99a32f7eddf29aeb4089c03ecfeac1906342cc6d2a858809b431b486152e4"} Jul 28 20:10:49.844244 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:49.844025 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" Jul 28 20:10:49.893596 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:10:49.893554 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" podStartSLOduration=1.893528001 podStartE2EDuration="1.893528001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:10:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:10:49.893499149 +0000 UTC m=+2045.600711043" watchObservedRunningTime="2026-07-28 20:10:49.893528001 +0000 UTC m=+2045.600739895" Jul 28 20:11:00.851654 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:00.851603 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" Jul 28 20:11:02.833908 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:02.833873 2564 patch_prober.go:28] interesting pod/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 28 20:11:02.834311 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:02.833938 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" podUID="c2594252-5835-4722-a7c2-0280ac188eb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 28 20:11:02.911764 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:02.911730 2564 generic.go:356] "Generic (PLEG): container finished" podID="c2594252-5835-4722-a7c2-0280ac188eb6" containerID="330a0e80b8bbb7ce773b8639ea4fc16c363b4b880b32807cab63e728840afb99" exitCode=0 Jul 28 20:11:02.911922 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:02.911803 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" event={"ID":"c2594252-5835-4722-a7c2-0280ac188eb6","Type":"ContainerDied","Data":"330a0e80b8bbb7ce773b8639ea4fc16c363b4b880b32807cab63e728840afb99"} Jul 28 20:11:02.912307 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:02.912288 2564 scope.go:117] "RemoveContainer" containerID="330a0e80b8bbb7ce773b8639ea4fc16c363b4b880b32807cab63e728840afb99" Jul 28 20:11:02.913504 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:02.913479 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:11:02.913564 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:02.913549 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:11:02.913601 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:02.913587 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:11:03.918234 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:03.918204 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" event={"ID":"c2594252-5835-4722-a7c2-0280ac188eb6","Type":"ContainerStarted","Data":"6ecb325e85c1bac98da7f0a15d5be98dda1f09c855b2c2df0d88b51e23dc55ce"} Jul 28 20:11:04.893554 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:04.893506 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:11:04.893807 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:04.893721 2564 patch_prober.go:28] interesting pod/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 28 20:11:04.893807 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:04.893772 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" podUID="c2594252-5835-4722-a7c2-0280ac188eb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 28 20:11:05.894237 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:05.894197 2564 patch_prober.go:28] interesting pod/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 28 20:11:05.894673 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:05.894259 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" podUID="c2594252-5835-4722-a7c2-0280ac188eb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 28 20:11:06.893348 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:06.893311 2564 patch_prober.go:28] interesting pod/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 28 20:11:06.893532 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:06.893382 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" podUID="c2594252-5835-4722-a7c2-0280ac188eb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 28 20:11:07.897681 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:07.897642 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:11:07.898080 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:07.897971 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:11:07.898529 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:11:07.898510 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:12:24.045880 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.045835 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q"] Jul 28 20:12:24.046441 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.046122 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" podUID="4ffc65e2-ed14-4d33-92e1-25fd969b10c0" containerName="payload-processing" containerID="cri-o://2d2d8e6506f786f8909c248ea50cd90251ad6d117bc33e995387df0a2151efcd" gracePeriod=30 Jul 28 20:12:24.316348 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.316317 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" Jul 28 20:12:24.327698 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.327666 2564 generic.go:356] "Generic (PLEG): container finished" podID="4ffc65e2-ed14-4d33-92e1-25fd969b10c0" containerID="2d2d8e6506f786f8909c248ea50cd90251ad6d117bc33e995387df0a2151efcd" exitCode=0 Jul 28 20:12:24.327860 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.327701 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" Jul 28 20:12:24.327860 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.327738 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" event={"ID":"4ffc65e2-ed14-4d33-92e1-25fd969b10c0","Type":"ContainerDied","Data":"2d2d8e6506f786f8909c248ea50cd90251ad6d117bc33e995387df0a2151efcd"} Jul 28 20:12:24.327860 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.327798 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q" event={"ID":"4ffc65e2-ed14-4d33-92e1-25fd969b10c0","Type":"ContainerDied","Data":"faf99a32f7eddf29aeb4089c03ecfeac1906342cc6d2a858809b431b486152e4"} Jul 28 20:12:24.327860 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.327828 2564 scope.go:117] "RemoveContainer" containerID="2d2d8e6506f786f8909c248ea50cd90251ad6d117bc33e995387df0a2151efcd" Jul 28 20:12:24.335991 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.335970 2564 scope.go:117] "RemoveContainer" containerID="2d2d8e6506f786f8909c248ea50cd90251ad6d117bc33e995387df0a2151efcd" Jul 28 20:12:24.336255 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:12:24.336238 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d2d8e6506f786f8909c248ea50cd90251ad6d117bc33e995387df0a2151efcd\": container with ID starting with 2d2d8e6506f786f8909c248ea50cd90251ad6d117bc33e995387df0a2151efcd not found: ID does not exist" containerID="2d2d8e6506f786f8909c248ea50cd90251ad6d117bc33e995387df0a2151efcd" Jul 28 20:12:24.336314 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.336264 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d2d8e6506f786f8909c248ea50cd90251ad6d117bc33e995387df0a2151efcd"} err="failed to get container status \"2d2d8e6506f786f8909c248ea50cd90251ad6d117bc33e995387df0a2151efcd\": rpc error: code = NotFound desc = could not find container \"2d2d8e6506f786f8909c248ea50cd90251ad6d117bc33e995387df0a2151efcd\": container with ID starting with 2d2d8e6506f786f8909c248ea50cd90251ad6d117bc33e995387df0a2151efcd not found: ID does not exist" Jul 28 20:12:24.408103 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.408070 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ffc65e2-ed14-4d33-92e1-25fd969b10c0-plugins-config-volume\") pod \"4ffc65e2-ed14-4d33-92e1-25fd969b10c0\" (UID: \"4ffc65e2-ed14-4d33-92e1-25fd969b10c0\") " Jul 28 20:12:24.408290 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.408121 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9l76\" (UniqueName: \"kubernetes.io/projected/4ffc65e2-ed14-4d33-92e1-25fd969b10c0-kube-api-access-x9l76\") pod \"4ffc65e2-ed14-4d33-92e1-25fd969b10c0\" (UID: \"4ffc65e2-ed14-4d33-92e1-25fd969b10c0\") " Jul 28 20:12:24.408474 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.408449 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ffc65e2-ed14-4d33-92e1-25fd969b10c0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4ffc65e2-ed14-4d33-92e1-25fd969b10c0" (UID: "4ffc65e2-ed14-4d33-92e1-25fd969b10c0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:12:24.410282 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.410259 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ffc65e2-ed14-4d33-92e1-25fd969b10c0-kube-api-access-x9l76" (OuterVolumeSpecName: "kube-api-access-x9l76") pod "4ffc65e2-ed14-4d33-92e1-25fd969b10c0" (UID: "4ffc65e2-ed14-4d33-92e1-25fd969b10c0"). InnerVolumeSpecName "kube-api-access-x9l76". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:12:24.508921 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.508885 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ffc65e2-ed14-4d33-92e1-25fd969b10c0-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:12:24.508921 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.508916 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9l76\" (UniqueName: \"kubernetes.io/projected/4ffc65e2-ed14-4d33-92e1-25fd969b10c0-kube-api-access-x9l76\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:12:24.659018 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.652827 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q"] Jul 28 20:12:24.659018 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.658777 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d517388f-5948bf758b-6hd9q"] Jul 28 20:12:24.918587 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:24.918505 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ffc65e2-ed14-4d33-92e1-25fd969b10c0" path="/var/lib/kubelet/pods/4ffc65e2-ed14-4d33-92e1-25fd969b10c0/volumes" Jul 28 20:12:45.034282 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:12:45.034247 2564 scope.go:117] "RemoveContainer" containerID="be8587e72477f3f276882e72a93bbace964e59841ea77a2d4ab501d65cb469b2" Jul 28 20:13:17.781677 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:17.781613 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5"] Jul 28 20:13:17.782316 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:17.781907 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" podUID="c2594252-5835-4722-a7c2-0280ac188eb6" containerName="istio-proxy" containerID="cri-o://6ecb325e85c1bac98da7f0a15d5be98dda1f09c855b2c2df0d88b51e23dc55ce" gracePeriod=30 Jul 28 20:13:22.899801 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:22.899773 2564 patch_prober.go:28] interesting pod/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 28 20:13:22.900031 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:22.899829 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" podUID="c2594252-5835-4722-a7c2-0280ac188eb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 28 20:13:23.038545 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.038520 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:13:23.089214 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.089182 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vklwn\" (UniqueName: \"kubernetes.io/projected/c2594252-5835-4722-a7c2-0280ac188eb6-kube-api-access-vklwn\") pod \"c2594252-5835-4722-a7c2-0280ac188eb6\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " Jul 28 20:13:23.089411 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.089253 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-credential-socket\") pod \"c2594252-5835-4722-a7c2-0280ac188eb6\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " Jul 28 20:13:23.089411 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.089286 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-workload-socket\") pod \"c2594252-5835-4722-a7c2-0280ac188eb6\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " Jul 28 20:13:23.089411 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.089318 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-istio-envoy\") pod \"c2594252-5835-4722-a7c2-0280ac188eb6\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " Jul 28 20:13:23.089411 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.089346 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-istio-data\") pod \"c2594252-5835-4722-a7c2-0280ac188eb6\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " Jul 28 20:13:23.089411 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.089379 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2594252-5835-4722-a7c2-0280ac188eb6-istiod-ca-cert\") pod \"c2594252-5835-4722-a7c2-0280ac188eb6\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " Jul 28 20:13:23.089712 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.089424 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-workload-certs\") pod \"c2594252-5835-4722-a7c2-0280ac188eb6\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " Jul 28 20:13:23.089712 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.089465 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2594252-5835-4722-a7c2-0280ac188eb6-istio-token\") pod \"c2594252-5835-4722-a7c2-0280ac188eb6\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " Jul 28 20:13:23.089712 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.089515 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2594252-5835-4722-a7c2-0280ac188eb6-istio-podinfo\") pod \"c2594252-5835-4722-a7c2-0280ac188eb6\" (UID: \"c2594252-5835-4722-a7c2-0280ac188eb6\") " Jul 28 20:13:23.089712 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.089657 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c2594252-5835-4722-a7c2-0280ac188eb6" (UID: "c2594252-5835-4722-a7c2-0280ac188eb6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:13:23.089929 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.089826 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c2594252-5835-4722-a7c2-0280ac188eb6" (UID: "c2594252-5835-4722-a7c2-0280ac188eb6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:13:23.089929 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.089630 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c2594252-5835-4722-a7c2-0280ac188eb6" (UID: "c2594252-5835-4722-a7c2-0280ac188eb6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:13:23.089929 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.089849 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2594252-5835-4722-a7c2-0280ac188eb6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c2594252-5835-4722-a7c2-0280ac188eb6" (UID: "c2594252-5835-4722-a7c2-0280ac188eb6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:13:23.089929 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.089920 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-workload-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:13:23.090334 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.090309 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-istio-data" (OuterVolumeSpecName: "istio-data") pod "c2594252-5835-4722-a7c2-0280ac188eb6" (UID: "c2594252-5835-4722-a7c2-0280ac188eb6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:13:23.091579 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.091551 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2594252-5835-4722-a7c2-0280ac188eb6-kube-api-access-vklwn" (OuterVolumeSpecName: "kube-api-access-vklwn") pod "c2594252-5835-4722-a7c2-0280ac188eb6" (UID: "c2594252-5835-4722-a7c2-0280ac188eb6"). InnerVolumeSpecName "kube-api-access-vklwn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:13:23.091684 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.091671 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c2594252-5835-4722-a7c2-0280ac188eb6" (UID: "c2594252-5835-4722-a7c2-0280ac188eb6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:13:23.092089 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.092061 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2594252-5835-4722-a7c2-0280ac188eb6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c2594252-5835-4722-a7c2-0280ac188eb6" (UID: "c2594252-5835-4722-a7c2-0280ac188eb6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 20:13:23.092089 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.092062 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2594252-5835-4722-a7c2-0280ac188eb6-istio-token" (OuterVolumeSpecName: "istio-token") pod "c2594252-5835-4722-a7c2-0280ac188eb6" (UID: "c2594252-5835-4722-a7c2-0280ac188eb6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:13:23.191160 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.191119 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-istio-envoy\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:13:23.191160 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.191149 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-istio-data\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:13:23.191160 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.191160 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2594252-5835-4722-a7c2-0280ac188eb6-istiod-ca-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:13:23.191160 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.191168 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-workload-certs\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:13:23.191160 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.191176 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2594252-5835-4722-a7c2-0280ac188eb6-istio-token\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:13:23.191512 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.191184 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2594252-5835-4722-a7c2-0280ac188eb6-istio-podinfo\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:13:23.191512 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.191193 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vklwn\" (UniqueName: \"kubernetes.io/projected/c2594252-5835-4722-a7c2-0280ac188eb6-kube-api-access-vklwn\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:13:23.191512 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.191202 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2594252-5835-4722-a7c2-0280ac188eb6-credential-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:13:23.618312 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.618275 2564 generic.go:356] "Generic (PLEG): container finished" podID="c2594252-5835-4722-a7c2-0280ac188eb6" containerID="6ecb325e85c1bac98da7f0a15d5be98dda1f09c855b2c2df0d88b51e23dc55ce" exitCode=0 Jul 28 20:13:23.618515 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.618338 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" Jul 28 20:13:23.618515 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.618358 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" event={"ID":"c2594252-5835-4722-a7c2-0280ac188eb6","Type":"ContainerDied","Data":"6ecb325e85c1bac98da7f0a15d5be98dda1f09c855b2c2df0d88b51e23dc55ce"} Jul 28 20:13:23.618515 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.618417 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5" event={"ID":"c2594252-5835-4722-a7c2-0280ac188eb6","Type":"ContainerDied","Data":"d8225b4279746dcd86c876699579e94742ea1f579632d01e1a3d968c6c31cdfa"} Jul 28 20:13:23.618515 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.618445 2564 scope.go:117] "RemoveContainer" containerID="6ecb325e85c1bac98da7f0a15d5be98dda1f09c855b2c2df0d88b51e23dc55ce" Jul 28 20:13:23.628563 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.628539 2564 scope.go:117] "RemoveContainer" containerID="330a0e80b8bbb7ce773b8639ea4fc16c363b4b880b32807cab63e728840afb99" Jul 28 20:13:23.641438 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.641414 2564 scope.go:117] "RemoveContainer" containerID="6ecb325e85c1bac98da7f0a15d5be98dda1f09c855b2c2df0d88b51e23dc55ce" Jul 28 20:13:23.641828 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:13:23.641802 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ecb325e85c1bac98da7f0a15d5be98dda1f09c855b2c2df0d88b51e23dc55ce\": container with ID starting with 6ecb325e85c1bac98da7f0a15d5be98dda1f09c855b2c2df0d88b51e23dc55ce not found: ID does not exist" containerID="6ecb325e85c1bac98da7f0a15d5be98dda1f09c855b2c2df0d88b51e23dc55ce" Jul 28 20:13:23.641948 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.641843 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ecb325e85c1bac98da7f0a15d5be98dda1f09c855b2c2df0d88b51e23dc55ce"} err="failed to get container status \"6ecb325e85c1bac98da7f0a15d5be98dda1f09c855b2c2df0d88b51e23dc55ce\": rpc error: code = NotFound desc = could not find container \"6ecb325e85c1bac98da7f0a15d5be98dda1f09c855b2c2df0d88b51e23dc55ce\": container with ID starting with 6ecb325e85c1bac98da7f0a15d5be98dda1f09c855b2c2df0d88b51e23dc55ce not found: ID does not exist" Jul 28 20:13:23.641948 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.641869 2564 scope.go:117] "RemoveContainer" containerID="330a0e80b8bbb7ce773b8639ea4fc16c363b4b880b32807cab63e728840afb99" Jul 28 20:13:23.642174 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:13:23.642154 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"330a0e80b8bbb7ce773b8639ea4fc16c363b4b880b32807cab63e728840afb99\": container with ID starting with 330a0e80b8bbb7ce773b8639ea4fc16c363b4b880b32807cab63e728840afb99 not found: ID does not exist" containerID="330a0e80b8bbb7ce773b8639ea4fc16c363b4b880b32807cab63e728840afb99" Jul 28 20:13:23.642223 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.642181 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"330a0e80b8bbb7ce773b8639ea4fc16c363b4b880b32807cab63e728840afb99"} err="failed to get container status \"330a0e80b8bbb7ce773b8639ea4fc16c363b4b880b32807cab63e728840afb99\": rpc error: code = NotFound desc = could not find container \"330a0e80b8bbb7ce773b8639ea4fc16c363b4b880b32807cab63e728840afb99\": container with ID starting with 330a0e80b8bbb7ce773b8639ea4fc16c363b4b880b32807cab63e728840afb99 not found: ID does not exist" Jul 28 20:13:23.644407 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.644378 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5"] Jul 28 20:13:23.647554 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:23.647526 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d517388f-openshift-default-dc86887f6-dbzf5"] Jul 28 20:13:24.918475 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:13:24.918434 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2594252-5835-4722-a7c2-0280ac188eb6" path="/var/lib/kubelet/pods/c2594252-5835-4722-a7c2-0280ac188eb6/volumes" Jul 28 20:14:14.107161 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.107083 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r"] Jul 28 20:14:14.107618 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.107481 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2594252-5835-4722-a7c2-0280ac188eb6" containerName="istio-proxy" Jul 28 20:14:14.107618 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.107493 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2594252-5835-4722-a7c2-0280ac188eb6" containerName="istio-proxy" Jul 28 20:14:14.107618 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.107507 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ffc65e2-ed14-4d33-92e1-25fd969b10c0" containerName="payload-processing" Jul 28 20:14:14.107618 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.107513 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ffc65e2-ed14-4d33-92e1-25fd969b10c0" containerName="payload-processing" Jul 28 20:14:14.107618 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.107521 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2594252-5835-4722-a7c2-0280ac188eb6" containerName="istio-proxy" Jul 28 20:14:14.107618 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.107526 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2594252-5835-4722-a7c2-0280ac188eb6" containerName="istio-proxy" Jul 28 20:14:14.107618 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.107593 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2594252-5835-4722-a7c2-0280ac188eb6" containerName="istio-proxy" Jul 28 20:14:14.107618 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.107601 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ffc65e2-ed14-4d33-92e1-25fd969b10c0" containerName="payload-processing" Jul 28 20:14:14.110727 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.110709 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.113764 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.113746 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-24876e-openshift-default-dockercfg-zhpvc\"" Jul 28 20:14:14.125259 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.125233 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r"] Jul 28 20:14:14.189902 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.189863 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e27a9b9e-9536-428e-a861-3e61c6780049-istiod-ca-cert\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.189902 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.189911 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e27a9b9e-9536-428e-a861-3e61c6780049-istio-podinfo\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.190185 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.189948 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-istio-envoy\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.190185 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.189996 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-workload-certs\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.190185 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.190024 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e27a9b9e-9536-428e-a861-3e61c6780049-istio-token\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.190185 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.190087 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-workload-socket\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.190185 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.190121 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgzl4\" (UniqueName: \"kubernetes.io/projected/e27a9b9e-9536-428e-a861-3e61c6780049-kube-api-access-wgzl4\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.190185 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.190168 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-istio-data\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.190417 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.190191 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-credential-socket\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.291687 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.291653 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-workload-socket\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.291894 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.291705 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgzl4\" (UniqueName: \"kubernetes.io/projected/e27a9b9e-9536-428e-a861-3e61c6780049-kube-api-access-wgzl4\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.291894 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.291741 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-istio-data\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.291894 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.291759 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-credential-socket\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.291894 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.291786 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e27a9b9e-9536-428e-a861-3e61c6780049-istiod-ca-cert\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.291894 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.291807 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e27a9b9e-9536-428e-a861-3e61c6780049-istio-podinfo\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.291894 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.291845 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-istio-envoy\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.291894 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.291889 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-workload-certs\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.292280 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.291922 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e27a9b9e-9536-428e-a861-3e61c6780049-istio-token\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.292280 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.292070 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-workload-socket\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.292280 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.292194 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-credential-socket\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.292280 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.292266 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-workload-certs\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.292492 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.292439 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-istio-data\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.292554 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.292535 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e27a9b9e-9536-428e-a861-3e61c6780049-istiod-ca-cert\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.294239 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.294218 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-istio-envoy\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.294499 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.294479 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e27a9b9e-9536-428e-a861-3e61c6780049-istio-podinfo\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.301505 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.301481 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e27a9b9e-9536-428e-a861-3e61c6780049-istio-token\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.301621 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.301483 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgzl4\" (UniqueName: \"kubernetes.io/projected/e27a9b9e-9536-428e-a861-3e61c6780049-kube-api-access-wgzl4\") pod \"e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.421198 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.421096 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:14.558259 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.558230 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r"] Jul 28 20:14:14.560394 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:14:14.560360 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode27a9b9e_9536_428e_a861_3e61c6780049.slice/crio-e4109db9a01e499e218a56dbfe13e8a733281d548581d1f83e833e0d0b4f5bda WatchSource:0}: Error finding container e4109db9a01e499e218a56dbfe13e8a733281d548581d1f83e833e0d0b4f5bda: Status 404 returned error can't find the container with id e4109db9a01e499e218a56dbfe13e8a733281d548581d1f83e833e0d0b4f5bda Jul 28 20:14:14.562686 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.562648 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:14:14.562759 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.562720 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:14:14.562759 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.562746 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:14:14.875541 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.875508 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" event={"ID":"e27a9b9e-9536-428e-a861-3e61c6780049","Type":"ContainerStarted","Data":"d146222fd718351b54aa002f888cb58c46cdb7517cba3496030b60e8cfffe265"} Jul 28 20:14:14.875740 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.875557 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" event={"ID":"e27a9b9e-9536-428e-a861-3e61c6780049","Type":"ContainerStarted","Data":"e4109db9a01e499e218a56dbfe13e8a733281d548581d1f83e833e0d0b4f5bda"} Jul 28 20:14:14.895962 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:14.895901 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" podStartSLOduration=0.895877629 podStartE2EDuration="895.877629ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:14:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:14:14.895207965 +0000 UTC m=+2250.602419860" watchObservedRunningTime="2026-07-28 20:14:14.895877629 +0000 UTC m=+2250.603089523" Jul 28 20:14:15.422075 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:15.422037 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:15.427102 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:15.427077 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:15.880604 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:15.880569 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:15.881556 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:15.881537 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:17.706871 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.706832 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8"] Jul 28 20:14:17.707280 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.707267 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2594252-5835-4722-a7c2-0280ac188eb6" containerName="istio-proxy" Jul 28 20:14:17.715795 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.715765 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" Jul 28 20:14:17.719942 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.719915 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-24876e\"" Jul 28 20:14:17.721923 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.721900 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-24876e-dockercfg-cxn2q\"" Jul 28 20:14:17.727405 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.727374 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8"] Jul 28 20:14:17.744575 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.744545 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt"] Jul 28 20:14:17.748072 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.748056 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" Jul 28 20:14:17.757011 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.756981 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt"] Jul 28 20:14:17.831547 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.831505 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9mft\" (UniqueName: \"kubernetes.io/projected/10ea4263-8fc5-42f4-a9bf-4415450e1e5f-kube-api-access-q9mft\") pod \"payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8\" (UID: \"10ea4263-8fc5-42f4-a9bf-4415450e1e5f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" Jul 28 20:14:17.831760 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.831564 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10ea4263-8fc5-42f4-a9bf-4415450e1e5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8\" (UID: \"10ea4263-8fc5-42f4-a9bf-4415450e1e5f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" Jul 28 20:14:17.831760 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.831666 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxk65\" (UniqueName: \"kubernetes.io/projected/76cf1231-92ac-47e1-a86b-903ac067afec-kube-api-access-qxk65\") pod \"payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt\" (UID: \"76cf1231-92ac-47e1-a86b-903ac067afec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" Jul 28 20:14:17.831760 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.831695 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76cf1231-92ac-47e1-a86b-903ac067afec-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt\" (UID: \"76cf1231-92ac-47e1-a86b-903ac067afec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" Jul 28 20:14:17.932731 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.932695 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxk65\" (UniqueName: \"kubernetes.io/projected/76cf1231-92ac-47e1-a86b-903ac067afec-kube-api-access-qxk65\") pod \"payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt\" (UID: \"76cf1231-92ac-47e1-a86b-903ac067afec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" Jul 28 20:14:17.932731 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.932738 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76cf1231-92ac-47e1-a86b-903ac067afec-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt\" (UID: \"76cf1231-92ac-47e1-a86b-903ac067afec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" Jul 28 20:14:17.932975 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.932798 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9mft\" (UniqueName: \"kubernetes.io/projected/10ea4263-8fc5-42f4-a9bf-4415450e1e5f-kube-api-access-q9mft\") pod \"payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8\" (UID: \"10ea4263-8fc5-42f4-a9bf-4415450e1e5f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" Jul 28 20:14:17.932975 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.932829 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10ea4263-8fc5-42f4-a9bf-4415450e1e5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8\" (UID: \"10ea4263-8fc5-42f4-a9bf-4415450e1e5f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" Jul 28 20:14:17.933522 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.933504 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10ea4263-8fc5-42f4-a9bf-4415450e1e5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8\" (UID: \"10ea4263-8fc5-42f4-a9bf-4415450e1e5f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" Jul 28 20:14:17.933575 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.933509 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76cf1231-92ac-47e1-a86b-903ac067afec-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt\" (UID: \"76cf1231-92ac-47e1-a86b-903ac067afec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" Jul 28 20:14:17.941548 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.941518 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9mft\" (UniqueName: \"kubernetes.io/projected/10ea4263-8fc5-42f4-a9bf-4415450e1e5f-kube-api-access-q9mft\") pod \"payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8\" (UID: \"10ea4263-8fc5-42f4-a9bf-4415450e1e5f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" Jul 28 20:14:17.941967 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:17.941949 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxk65\" (UniqueName: \"kubernetes.io/projected/76cf1231-92ac-47e1-a86b-903ac067afec-kube-api-access-qxk65\") pod \"payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt\" (UID: \"76cf1231-92ac-47e1-a86b-903ac067afec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" Jul 28 20:14:18.028994 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:18.028901 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" Jul 28 20:14:18.060524 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:18.060493 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" Jul 28 20:14:18.183153 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:18.183120 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8"] Jul 28 20:14:18.186655 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:14:18.186611 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ea4263_8fc5_42f4_a9bf_4415450e1e5f.slice/crio-aaf17964b4dd32aaa58964c417e43ea41d1d6fa1c184b611d21cfc8b94e374b6 WatchSource:0}: Error finding container aaf17964b4dd32aaa58964c417e43ea41d1d6fa1c184b611d21cfc8b94e374b6: Status 404 returned error can't find the container with id aaf17964b4dd32aaa58964c417e43ea41d1d6fa1c184b611d21cfc8b94e374b6 Jul 28 20:14:18.217711 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:18.217684 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt"] Jul 28 20:14:18.227868 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:14:18.227834 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76cf1231_92ac_47e1_a86b_903ac067afec.slice/crio-c492f653e5798af57422da1981b287a8ee0565d516fc9f260dfd043c76a08d5e WatchSource:0}: Error finding container c492f653e5798af57422da1981b287a8ee0565d516fc9f260dfd043c76a08d5e: Status 404 returned error can't find the container with id c492f653e5798af57422da1981b287a8ee0565d516fc9f260dfd043c76a08d5e Jul 28 20:14:18.898240 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:18.898201 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" event={"ID":"76cf1231-92ac-47e1-a86b-903ac067afec","Type":"ContainerStarted","Data":"3301e3c8bd306ad08a0adba9caabdc022f54f4212b49f9ab87c7313471d244a1"} Jul 28 20:14:18.898765 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:18.898260 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" event={"ID":"76cf1231-92ac-47e1-a86b-903ac067afec","Type":"ContainerStarted","Data":"c492f653e5798af57422da1981b287a8ee0565d516fc9f260dfd043c76a08d5e"} Jul 28 20:14:18.898765 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:18.898298 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" Jul 28 20:14:18.899552 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:18.899530 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" event={"ID":"10ea4263-8fc5-42f4-a9bf-4415450e1e5f","Type":"ContainerStarted","Data":"279844bb5ae93b6fae5e7823f1f999fedab5ac71501623eb6dbfca176e10d407"} Jul 28 20:14:18.899667 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:18.899573 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" event={"ID":"10ea4263-8fc5-42f4-a9bf-4415450e1e5f","Type":"ContainerStarted","Data":"aaf17964b4dd32aaa58964c417e43ea41d1d6fa1c184b611d21cfc8b94e374b6"} Jul 28 20:14:18.899720 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:18.899669 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" Jul 28 20:14:18.920498 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:18.920449 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" podStartSLOduration=1.920435954 podStartE2EDuration="1.920435954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:14:18.918654881 +0000 UTC m=+2254.625866768" watchObservedRunningTime="2026-07-28 20:14:18.920435954 +0000 UTC m=+2254.627647837" Jul 28 20:14:18.936341 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:18.936289 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" podStartSLOduration=1.936272304 podStartE2EDuration="1.936272304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:14:18.935375038 +0000 UTC m=+2254.642586955" watchObservedRunningTime="2026-07-28 20:14:18.936272304 +0000 UTC m=+2254.643484196" Jul 28 20:14:29.907776 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:29.907739 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" Jul 28 20:14:29.908321 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:29.907811 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" Jul 28 20:14:30.882068 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:30.882032 2564 patch_prober.go:28] interesting pod/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 28 20:14:30.882264 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:30.882093 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" podUID="e27a9b9e-9536-428e-a861-3e61c6780049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 28 20:14:30.962393 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:30.962360 2564 generic.go:356] "Generic (PLEG): container finished" podID="e27a9b9e-9536-428e-a861-3e61c6780049" containerID="d146222fd718351b54aa002f888cb58c46cdb7517cba3496030b60e8cfffe265" exitCode=0 Jul 28 20:14:30.962393 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:30.962409 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" event={"ID":"e27a9b9e-9536-428e-a861-3e61c6780049","Type":"ContainerDied","Data":"d146222fd718351b54aa002f888cb58c46cdb7517cba3496030b60e8cfffe265"} Jul 28 20:14:30.962855 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:30.962826 2564 scope.go:117] "RemoveContainer" containerID="d146222fd718351b54aa002f888cb58c46cdb7517cba3496030b60e8cfffe265" Jul 28 20:14:30.963855 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:30.963831 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:14:30.963920 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:30.963898 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:14:30.963961 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:30.963934 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:14:31.970262 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:31.970228 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" event={"ID":"e27a9b9e-9536-428e-a861-3e61c6780049","Type":"ContainerStarted","Data":"f9bb4268c9ca6329de55a29989afd6f3e0ac1ec798a701c359ed72b5fc717743"} Jul 28 20:14:32.421577 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:32.421538 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:32.421770 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:32.421736 2564 patch_prober.go:28] interesting pod/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 28 20:14:32.421818 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:32.421782 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" podUID="e27a9b9e-9536-428e-a861-3e61c6780049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 28 20:14:33.421919 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:33.421878 2564 patch_prober.go:28] interesting pod/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 28 20:14:33.422330 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:33.421941 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" podUID="e27a9b9e-9536-428e-a861-3e61c6780049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 28 20:14:34.422525 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:34.422489 2564 patch_prober.go:28] interesting pod/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 28 20:14:34.422963 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:34.422549 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" podUID="e27a9b9e-9536-428e-a861-3e61c6780049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 28 20:14:35.421880 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:35.421846 2564 patch_prober.go:28] interesting pod/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 28 20:14:35.422062 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:35.421926 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" podUID="e27a9b9e-9536-428e-a861-3e61c6780049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 28 20:14:36.426015 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:36.425982 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:36.426471 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:36.426380 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:36.427135 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:36.427119 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:14:46.417106 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:46.417042 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9"] Jul 28 20:14:46.420251 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:46.420225 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" Jul 28 20:14:46.423586 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:46.423563 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ab7491\"" Jul 28 20:14:46.423751 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:46.423563 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ab7491-dockercfg-t6zht\"" Jul 28 20:14:46.431951 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:46.431923 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9"] Jul 28 20:14:46.515801 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:46.515761 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqb7r\" (UniqueName: \"kubernetes.io/projected/06c952b4-9071-4ec6-9bc4-34ded1a98d2f-kube-api-access-qqb7r\") pod \"payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9\" (UID: \"06c952b4-9071-4ec6-9bc4-34ded1a98d2f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" Jul 28 20:14:46.515979 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:46.515840 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06c952b4-9071-4ec6-9bc4-34ded1a98d2f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9\" (UID: \"06c952b4-9071-4ec6-9bc4-34ded1a98d2f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" Jul 28 20:14:46.617127 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:46.617089 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqb7r\" (UniqueName: \"kubernetes.io/projected/06c952b4-9071-4ec6-9bc4-34ded1a98d2f-kube-api-access-qqb7r\") pod \"payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9\" (UID: \"06c952b4-9071-4ec6-9bc4-34ded1a98d2f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" Jul 28 20:14:46.617305 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:46.617169 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06c952b4-9071-4ec6-9bc4-34ded1a98d2f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9\" (UID: \"06c952b4-9071-4ec6-9bc4-34ded1a98d2f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" Jul 28 20:14:46.617846 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:46.617822 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06c952b4-9071-4ec6-9bc4-34ded1a98d2f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9\" (UID: \"06c952b4-9071-4ec6-9bc4-34ded1a98d2f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" Jul 28 20:14:46.626119 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:46.626088 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqb7r\" (UniqueName: \"kubernetes.io/projected/06c952b4-9071-4ec6-9bc4-34ded1a98d2f-kube-api-access-qqb7r\") pod \"payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9\" (UID: \"06c952b4-9071-4ec6-9bc4-34ded1a98d2f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" Jul 28 20:14:46.734606 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:46.734523 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" Jul 28 20:14:46.872842 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:46.872817 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9"] Jul 28 20:14:46.875038 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:14:46.875008 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06c952b4_9071_4ec6_9bc4_34ded1a98d2f.slice/crio-56bd04d7ff076a19e4fab626973f3da7bc4dfea9fc11009224a28c330d4de872 WatchSource:0}: Error finding container 56bd04d7ff076a19e4fab626973f3da7bc4dfea9fc11009224a28c330d4de872: Status 404 returned error can't find the container with id 56bd04d7ff076a19e4fab626973f3da7bc4dfea9fc11009224a28c330d4de872 Jul 28 20:14:47.055829 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:47.055783 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" event={"ID":"06c952b4-9071-4ec6-9bc4-34ded1a98d2f","Type":"ContainerStarted","Data":"cf10065e766faa25610df74f5f4e951a74a1a4d3f570e954a2813dfed30cdfd1"} Jul 28 20:14:47.056025 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:47.055895 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" event={"ID":"06c952b4-9071-4ec6-9bc4-34ded1a98d2f","Type":"ContainerStarted","Data":"56bd04d7ff076a19e4fab626973f3da7bc4dfea9fc11009224a28c330d4de872"} Jul 28 20:14:47.056025 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:47.055931 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" Jul 28 20:14:47.079318 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:47.079265 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" podStartSLOduration=1.079250042 podStartE2EDuration="1.079250042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:14:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:14:47.077450447 +0000 UTC m=+2282.784662362" watchObservedRunningTime="2026-07-28 20:14:47.079250042 +0000 UTC m=+2282.786461935" Jul 28 20:14:58.063379 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:14:58.063348 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" Jul 28 20:16:36.460931 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.460896 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt"] Jul 28 20:16:36.461495 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.461212 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" podUID="76cf1231-92ac-47e1-a86b-903ac067afec" containerName="payload-processing" containerID="cri-o://3301e3c8bd306ad08a0adba9caabdc022f54f4212b49f9ab87c7313471d244a1" gracePeriod=30 Jul 28 20:16:36.466406 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.466377 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8"] Jul 28 20:16:36.468320 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.468270 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" podUID="10ea4263-8fc5-42f4-a9bf-4415450e1e5f" containerName="payload-pre-processing" containerID="cri-o://279844bb5ae93b6fae5e7823f1f999fedab5ac71501623eb6dbfca176e10d407" gracePeriod=30 Jul 28 20:16:36.609790 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.609760 2564 generic.go:356] "Generic (PLEG): container finished" podID="76cf1231-92ac-47e1-a86b-903ac067afec" containerID="3301e3c8bd306ad08a0adba9caabdc022f54f4212b49f9ab87c7313471d244a1" exitCode=0 Jul 28 20:16:36.610128 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.610102 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" event={"ID":"76cf1231-92ac-47e1-a86b-903ac067afec","Type":"ContainerDied","Data":"3301e3c8bd306ad08a0adba9caabdc022f54f4212b49f9ab87c7313471d244a1"} Jul 28 20:16:36.612144 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.612056 2564 generic.go:356] "Generic (PLEG): container finished" podID="10ea4263-8fc5-42f4-a9bf-4415450e1e5f" containerID="279844bb5ae93b6fae5e7823f1f999fedab5ac71501623eb6dbfca176e10d407" exitCode=0 Jul 28 20:16:36.612144 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.612104 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" event={"ID":"10ea4263-8fc5-42f4-a9bf-4415450e1e5f","Type":"ContainerDied","Data":"279844bb5ae93b6fae5e7823f1f999fedab5ac71501623eb6dbfca176e10d407"} Jul 28 20:16:36.734553 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.734521 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" Jul 28 20:16:36.771433 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.771408 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" Jul 28 20:16:36.810021 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.809990 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxk65\" (UniqueName: \"kubernetes.io/projected/76cf1231-92ac-47e1-a86b-903ac067afec-kube-api-access-qxk65\") pod \"76cf1231-92ac-47e1-a86b-903ac067afec\" (UID: \"76cf1231-92ac-47e1-a86b-903ac067afec\") " Jul 28 20:16:36.810184 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.810050 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76cf1231-92ac-47e1-a86b-903ac067afec-plugins-config-volume\") pod \"76cf1231-92ac-47e1-a86b-903ac067afec\" (UID: \"76cf1231-92ac-47e1-a86b-903ac067afec\") " Jul 28 20:16:36.810443 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.810417 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76cf1231-92ac-47e1-a86b-903ac067afec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "76cf1231-92ac-47e1-a86b-903ac067afec" (UID: "76cf1231-92ac-47e1-a86b-903ac067afec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:16:36.812137 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.812111 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76cf1231-92ac-47e1-a86b-903ac067afec-kube-api-access-qxk65" (OuterVolumeSpecName: "kube-api-access-qxk65") pod "76cf1231-92ac-47e1-a86b-903ac067afec" (UID: "76cf1231-92ac-47e1-a86b-903ac067afec"). InnerVolumeSpecName "kube-api-access-qxk65". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:16:36.911678 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.911629 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9mft\" (UniqueName: \"kubernetes.io/projected/10ea4263-8fc5-42f4-a9bf-4415450e1e5f-kube-api-access-q9mft\") pod \"10ea4263-8fc5-42f4-a9bf-4415450e1e5f\" (UID: \"10ea4263-8fc5-42f4-a9bf-4415450e1e5f\") " Jul 28 20:16:36.911850 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.911705 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10ea4263-8fc5-42f4-a9bf-4415450e1e5f-plugins-config-volume\") pod \"10ea4263-8fc5-42f4-a9bf-4415450e1e5f\" (UID: \"10ea4263-8fc5-42f4-a9bf-4415450e1e5f\") " Jul 28 20:16:36.912058 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.912032 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10ea4263-8fc5-42f4-a9bf-4415450e1e5f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "10ea4263-8fc5-42f4-a9bf-4415450e1e5f" (UID: "10ea4263-8fc5-42f4-a9bf-4415450e1e5f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:16:36.912058 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.912048 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76cf1231-92ac-47e1-a86b-903ac067afec-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:16:36.912182 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.912069 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxk65\" (UniqueName: \"kubernetes.io/projected/76cf1231-92ac-47e1-a86b-903ac067afec-kube-api-access-qxk65\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:16:36.913792 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:36.913769 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10ea4263-8fc5-42f4-a9bf-4415450e1e5f-kube-api-access-q9mft" (OuterVolumeSpecName: "kube-api-access-q9mft") pod "10ea4263-8fc5-42f4-a9bf-4415450e1e5f" (UID: "10ea4263-8fc5-42f4-a9bf-4415450e1e5f"). InnerVolumeSpecName "kube-api-access-q9mft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:16:37.013504 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:37.013414 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9mft\" (UniqueName: \"kubernetes.io/projected/10ea4263-8fc5-42f4-a9bf-4415450e1e5f-kube-api-access-q9mft\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:16:37.013504 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:37.013445 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10ea4263-8fc5-42f4-a9bf-4415450e1e5f-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:16:37.618585 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:37.618551 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" Jul 28 20:16:37.619074 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:37.618578 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8" event={"ID":"10ea4263-8fc5-42f4-a9bf-4415450e1e5f","Type":"ContainerDied","Data":"aaf17964b4dd32aaa58964c417e43ea41d1d6fa1c184b611d21cfc8b94e374b6"} Jul 28 20:16:37.619074 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:37.618673 2564 scope.go:117] "RemoveContainer" containerID="279844bb5ae93b6fae5e7823f1f999fedab5ac71501623eb6dbfca176e10d407" Jul 28 20:16:37.620029 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:37.619997 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" event={"ID":"76cf1231-92ac-47e1-a86b-903ac067afec","Type":"ContainerDied","Data":"c492f653e5798af57422da1981b287a8ee0565d516fc9f260dfd043c76a08d5e"} Jul 28 20:16:37.620097 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:37.620040 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt" Jul 28 20:16:37.627563 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:37.627545 2564 scope.go:117] "RemoveContainer" containerID="3301e3c8bd306ad08a0adba9caabdc022f54f4212b49f9ab87c7313471d244a1" Jul 28 20:16:37.640816 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:37.640791 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt"] Jul 28 20:16:37.644542 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:37.644515 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-24876e-7bdb55fd54-p6lqt"] Jul 28 20:16:37.656296 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:37.656272 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8"] Jul 28 20:16:37.660084 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:37.660060 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-24876e-85fb4f9d5b-s2hv8"] Jul 28 20:16:38.918898 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:38.918858 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10ea4263-8fc5-42f4-a9bf-4415450e1e5f" path="/var/lib/kubelet/pods/10ea4263-8fc5-42f4-a9bf-4415450e1e5f/volumes" Jul 28 20:16:38.919401 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:38.919377 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76cf1231-92ac-47e1-a86b-903ac067afec" path="/var/lib/kubelet/pods/76cf1231-92ac-47e1-a86b-903ac067afec/volumes" Jul 28 20:16:43.132171 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:43.132137 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r"] Jul 28 20:16:43.132583 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:43.132365 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" podUID="e27a9b9e-9536-428e-a861-3e61c6780049" containerName="istio-proxy" containerID="cri-o://f9bb4268c9ca6329de55a29989afd6f3e0ac1ec798a701c359ed72b5fc717743" gracePeriod=30 Jul 28 20:16:48.384772 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.384747 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:16:48.529845 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.529753 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgzl4\" (UniqueName: \"kubernetes.io/projected/e27a9b9e-9536-428e-a861-3e61c6780049-kube-api-access-wgzl4\") pod \"e27a9b9e-9536-428e-a861-3e61c6780049\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " Jul 28 20:16:48.529845 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.529801 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e27a9b9e-9536-428e-a861-3e61c6780049-istio-token\") pod \"e27a9b9e-9536-428e-a861-3e61c6780049\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " Jul 28 20:16:48.529845 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.529842 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-credential-socket\") pod \"e27a9b9e-9536-428e-a861-3e61c6780049\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " Jul 28 20:16:48.530139 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.530019 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e27a9b9e-9536-428e-a861-3e61c6780049-istio-podinfo\") pod \"e27a9b9e-9536-428e-a861-3e61c6780049\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " Jul 28 20:16:48.530139 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.530067 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-workload-certs\") pod \"e27a9b9e-9536-428e-a861-3e61c6780049\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " Jul 28 20:16:48.530139 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.530094 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-workload-socket\") pod \"e27a9b9e-9536-428e-a861-3e61c6780049\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " Jul 28 20:16:48.530139 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.530120 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-istio-envoy\") pod \"e27a9b9e-9536-428e-a861-3e61c6780049\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " Jul 28 20:16:48.530139 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.530129 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e27a9b9e-9536-428e-a861-3e61c6780049" (UID: "e27a9b9e-9536-428e-a861-3e61c6780049"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:16:48.530480 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.530147 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-istio-data\") pod \"e27a9b9e-9536-428e-a861-3e61c6780049\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " Jul 28 20:16:48.530480 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.530203 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e27a9b9e-9536-428e-a861-3e61c6780049-istiod-ca-cert\") pod \"e27a9b9e-9536-428e-a861-3e61c6780049\" (UID: \"e27a9b9e-9536-428e-a861-3e61c6780049\") " Jul 28 20:16:48.530480 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.530378 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e27a9b9e-9536-428e-a861-3e61c6780049" (UID: "e27a9b9e-9536-428e-a861-3e61c6780049"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:16:48.530681 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.530631 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-credential-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:16:48.530746 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.530690 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-workload-certs\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:16:48.530746 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.530703 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e27a9b9e-9536-428e-a861-3e61c6780049-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e27a9b9e-9536-428e-a861-3e61c6780049" (UID: "e27a9b9e-9536-428e-a861-3e61c6780049"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:16:48.530860 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.530770 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e27a9b9e-9536-428e-a861-3e61c6780049" (UID: "e27a9b9e-9536-428e-a861-3e61c6780049"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:16:48.531170 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.531060 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-istio-data" (OuterVolumeSpecName: "istio-data") pod "e27a9b9e-9536-428e-a861-3e61c6780049" (UID: "e27a9b9e-9536-428e-a861-3e61c6780049"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:16:48.532110 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.532086 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e27a9b9e-9536-428e-a861-3e61c6780049-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e27a9b9e-9536-428e-a861-3e61c6780049" (UID: "e27a9b9e-9536-428e-a861-3e61c6780049"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 20:16:48.532308 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.532294 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e27a9b9e-9536-428e-a861-3e61c6780049-istio-token" (OuterVolumeSpecName: "istio-token") pod "e27a9b9e-9536-428e-a861-3e61c6780049" (UID: "e27a9b9e-9536-428e-a861-3e61c6780049"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:16:48.532524 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.532498 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e27a9b9e-9536-428e-a861-3e61c6780049" (UID: "e27a9b9e-9536-428e-a861-3e61c6780049"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:16:48.532621 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.532527 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e27a9b9e-9536-428e-a861-3e61c6780049-kube-api-access-wgzl4" (OuterVolumeSpecName: "kube-api-access-wgzl4") pod "e27a9b9e-9536-428e-a861-3e61c6780049" (UID: "e27a9b9e-9536-428e-a861-3e61c6780049"). InnerVolumeSpecName "kube-api-access-wgzl4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:16:48.632104 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.632066 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgzl4\" (UniqueName: \"kubernetes.io/projected/e27a9b9e-9536-428e-a861-3e61c6780049-kube-api-access-wgzl4\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:16:48.632104 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.632097 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e27a9b9e-9536-428e-a861-3e61c6780049-istio-token\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:16:48.632104 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.632106 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e27a9b9e-9536-428e-a861-3e61c6780049-istio-podinfo\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:16:48.632348 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.632115 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-workload-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:16:48.632348 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.632128 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-istio-envoy\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:16:48.632348 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.632135 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e27a9b9e-9536-428e-a861-3e61c6780049-istio-data\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:16:48.632348 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.632143 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e27a9b9e-9536-428e-a861-3e61c6780049-istiod-ca-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:16:48.682453 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.682409 2564 generic.go:356] "Generic (PLEG): container finished" podID="e27a9b9e-9536-428e-a861-3e61c6780049" containerID="f9bb4268c9ca6329de55a29989afd6f3e0ac1ec798a701c359ed72b5fc717743" exitCode=0 Jul 28 20:16:48.682453 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.682457 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" Jul 28 20:16:48.682685 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.682494 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" event={"ID":"e27a9b9e-9536-428e-a861-3e61c6780049","Type":"ContainerDied","Data":"f9bb4268c9ca6329de55a29989afd6f3e0ac1ec798a701c359ed72b5fc717743"} Jul 28 20:16:48.682685 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.682549 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r" event={"ID":"e27a9b9e-9536-428e-a861-3e61c6780049","Type":"ContainerDied","Data":"e4109db9a01e499e218a56dbfe13e8a733281d548581d1f83e833e0d0b4f5bda"} Jul 28 20:16:48.682685 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.682572 2564 scope.go:117] "RemoveContainer" containerID="f9bb4268c9ca6329de55a29989afd6f3e0ac1ec798a701c359ed72b5fc717743" Jul 28 20:16:48.691647 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.691616 2564 scope.go:117] "RemoveContainer" containerID="d146222fd718351b54aa002f888cb58c46cdb7517cba3496030b60e8cfffe265" Jul 28 20:16:48.699390 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.699374 2564 scope.go:117] "RemoveContainer" containerID="f9bb4268c9ca6329de55a29989afd6f3e0ac1ec798a701c359ed72b5fc717743" Jul 28 20:16:48.699667 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:16:48.699625 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9bb4268c9ca6329de55a29989afd6f3e0ac1ec798a701c359ed72b5fc717743\": container with ID starting with f9bb4268c9ca6329de55a29989afd6f3e0ac1ec798a701c359ed72b5fc717743 not found: ID does not exist" containerID="f9bb4268c9ca6329de55a29989afd6f3e0ac1ec798a701c359ed72b5fc717743" Jul 28 20:16:48.699751 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.699679 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9bb4268c9ca6329de55a29989afd6f3e0ac1ec798a701c359ed72b5fc717743"} err="failed to get container status \"f9bb4268c9ca6329de55a29989afd6f3e0ac1ec798a701c359ed72b5fc717743\": rpc error: code = NotFound desc = could not find container \"f9bb4268c9ca6329de55a29989afd6f3e0ac1ec798a701c359ed72b5fc717743\": container with ID starting with f9bb4268c9ca6329de55a29989afd6f3e0ac1ec798a701c359ed72b5fc717743 not found: ID does not exist" Jul 28 20:16:48.699751 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.699703 2564 scope.go:117] "RemoveContainer" containerID="d146222fd718351b54aa002f888cb58c46cdb7517cba3496030b60e8cfffe265" Jul 28 20:16:48.699953 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:16:48.699935 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d146222fd718351b54aa002f888cb58c46cdb7517cba3496030b60e8cfffe265\": container with ID starting with d146222fd718351b54aa002f888cb58c46cdb7517cba3496030b60e8cfffe265 not found: ID does not exist" containerID="d146222fd718351b54aa002f888cb58c46cdb7517cba3496030b60e8cfffe265" Jul 28 20:16:48.699992 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.699958 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d146222fd718351b54aa002f888cb58c46cdb7517cba3496030b60e8cfffe265"} err="failed to get container status \"d146222fd718351b54aa002f888cb58c46cdb7517cba3496030b60e8cfffe265\": rpc error: code = NotFound desc = could not find container \"d146222fd718351b54aa002f888cb58c46cdb7517cba3496030b60e8cfffe265\": container with ID starting with d146222fd718351b54aa002f888cb58c46cdb7517cba3496030b60e8cfffe265 not found: ID does not exist" Jul 28 20:16:48.705486 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.705463 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r"] Jul 28 20:16:48.709863 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.709840 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-24876e-openshift-default-c664fdf66-htt8r"] Jul 28 20:16:48.918678 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:48.918619 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e27a9b9e-9536-428e-a861-3e61c6780049" path="/var/lib/kubelet/pods/e27a9b9e-9536-428e-a861-3e61c6780049/volumes" Jul 28 20:16:54.477856 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:54.477814 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9"] Jul 28 20:16:54.478233 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:54.478203 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" podUID="06c952b4-9071-4ec6-9bc4-34ded1a98d2f" containerName="payload-pre-processing" containerID="cri-o://cf10065e766faa25610df74f5f4e951a74a1a4d3f570e954a2813dfed30cdfd1" gracePeriod=30 Jul 28 20:16:54.717373 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:54.717337 2564 generic.go:356] "Generic (PLEG): container finished" podID="06c952b4-9071-4ec6-9bc4-34ded1a98d2f" containerID="cf10065e766faa25610df74f5f4e951a74a1a4d3f570e954a2813dfed30cdfd1" exitCode=0 Jul 28 20:16:54.717564 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:54.717414 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" event={"ID":"06c952b4-9071-4ec6-9bc4-34ded1a98d2f","Type":"ContainerDied","Data":"cf10065e766faa25610df74f5f4e951a74a1a4d3f570e954a2813dfed30cdfd1"} Jul 28 20:16:54.739619 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:54.739551 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" Jul 28 20:16:54.787145 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:54.787114 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06c952b4-9071-4ec6-9bc4-34ded1a98d2f-plugins-config-volume\") pod \"06c952b4-9071-4ec6-9bc4-34ded1a98d2f\" (UID: \"06c952b4-9071-4ec6-9bc4-34ded1a98d2f\") " Jul 28 20:16:54.787308 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:54.787161 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqb7r\" (UniqueName: \"kubernetes.io/projected/06c952b4-9071-4ec6-9bc4-34ded1a98d2f-kube-api-access-qqb7r\") pod \"06c952b4-9071-4ec6-9bc4-34ded1a98d2f\" (UID: \"06c952b4-9071-4ec6-9bc4-34ded1a98d2f\") " Jul 28 20:16:54.787518 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:54.787496 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06c952b4-9071-4ec6-9bc4-34ded1a98d2f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "06c952b4-9071-4ec6-9bc4-34ded1a98d2f" (UID: "06c952b4-9071-4ec6-9bc4-34ded1a98d2f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:16:54.789185 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:54.789159 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06c952b4-9071-4ec6-9bc4-34ded1a98d2f-kube-api-access-qqb7r" (OuterVolumeSpecName: "kube-api-access-qqb7r") pod "06c952b4-9071-4ec6-9bc4-34ded1a98d2f" (UID: "06c952b4-9071-4ec6-9bc4-34ded1a98d2f"). InnerVolumeSpecName "kube-api-access-qqb7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:16:54.887919 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:54.887886 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06c952b4-9071-4ec6-9bc4-34ded1a98d2f-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:16:54.887919 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:54.887913 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqb7r\" (UniqueName: \"kubernetes.io/projected/06c952b4-9071-4ec6-9bc4-34ded1a98d2f-kube-api-access-qqb7r\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:16:55.722861 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:55.722824 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" event={"ID":"06c952b4-9071-4ec6-9bc4-34ded1a98d2f","Type":"ContainerDied","Data":"56bd04d7ff076a19e4fab626973f3da7bc4dfea9fc11009224a28c330d4de872"} Jul 28 20:16:55.723314 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:55.722881 2564 scope.go:117] "RemoveContainer" containerID="cf10065e766faa25610df74f5f4e951a74a1a4d3f570e954a2813dfed30cdfd1" Jul 28 20:16:55.723314 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:55.722839 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9" Jul 28 20:16:55.759037 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:55.759006 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9"] Jul 28 20:16:55.768834 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:55.768810 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ab7491-857bc9bb7b-rvsf9"] Jul 28 20:16:56.918371 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:16:56.918336 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06c952b4-9071-4ec6-9bc4-34ded1a98d2f" path="/var/lib/kubelet/pods/06c952b4-9071-4ec6-9bc4-34ded1a98d2f/volumes" Jul 28 20:17:02.127473 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.127442 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn"] Jul 28 20:17:02.127900 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.127834 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e27a9b9e-9536-428e-a861-3e61c6780049" containerName="istio-proxy" Jul 28 20:17:02.127900 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.127846 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e27a9b9e-9536-428e-a861-3e61c6780049" containerName="istio-proxy" Jul 28 20:17:02.127900 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.127864 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76cf1231-92ac-47e1-a86b-903ac067afec" containerName="payload-processing" Jul 28 20:17:02.127900 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.127870 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="76cf1231-92ac-47e1-a86b-903ac067afec" containerName="payload-processing" Jul 28 20:17:02.127900 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.127880 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e27a9b9e-9536-428e-a861-3e61c6780049" containerName="istio-proxy" Jul 28 20:17:02.127900 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.127886 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e27a9b9e-9536-428e-a861-3e61c6780049" containerName="istio-proxy" Jul 28 20:17:02.127900 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.127896 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10ea4263-8fc5-42f4-a9bf-4415450e1e5f" containerName="payload-pre-processing" Jul 28 20:17:02.127900 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.127902 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="10ea4263-8fc5-42f4-a9bf-4415450e1e5f" containerName="payload-pre-processing" Jul 28 20:17:02.128221 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.127908 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06c952b4-9071-4ec6-9bc4-34ded1a98d2f" containerName="payload-pre-processing" Jul 28 20:17:02.128221 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.127914 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="06c952b4-9071-4ec6-9bc4-34ded1a98d2f" containerName="payload-pre-processing" Jul 28 20:17:02.128221 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.127971 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e27a9b9e-9536-428e-a861-3e61c6780049" containerName="istio-proxy" Jul 28 20:17:02.128221 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.128003 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="76cf1231-92ac-47e1-a86b-903ac067afec" containerName="payload-processing" Jul 28 20:17:02.128221 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.128008 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="06c952b4-9071-4ec6-9bc4-34ded1a98d2f" containerName="payload-pre-processing" Jul 28 20:17:02.128221 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.128016 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="10ea4263-8fc5-42f4-a9bf-4415450e1e5f" containerName="payload-pre-processing" Jul 28 20:17:02.132715 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.132692 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.136387 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.136363 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a5ccdf-openshift-default-dockercfg-7m8z7\"" Jul 28 20:17:02.146425 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.146398 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn"] Jul 28 20:17:02.255249 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.255214 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-envoy\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.255249 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.255263 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-workload-certs\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.255486 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.255283 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-podinfo\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.255486 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.255369 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istiod-ca-cert\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.255486 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.255413 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-credential-socket\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.255486 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.255462 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-workload-socket\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.255620 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.255496 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-data\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.255620 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.255514 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-token\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.255620 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.255545 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nfgs\" (UniqueName: \"kubernetes.io/projected/0457ed66-7e63-4bcf-a489-16c40ebc25a1-kube-api-access-6nfgs\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.356224 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.356184 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-envoy\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.356408 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.356233 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-workload-certs\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.356408 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.356354 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-podinfo\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.356408 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.356396 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istiod-ca-cert\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.356510 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.356428 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-credential-socket\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.356510 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.356477 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-workload-socket\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.356606 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.356514 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-data\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.356606 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.356544 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-token\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.356606 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.356576 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-workload-certs\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.356606 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.356593 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nfgs\" (UniqueName: \"kubernetes.io/projected/0457ed66-7e63-4bcf-a489-16c40ebc25a1-kube-api-access-6nfgs\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.356914 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.356881 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-data\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.357141 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.357117 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-workload-socket\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.357241 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.357150 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-credential-socket\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.357241 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.357165 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istiod-ca-cert\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.358816 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.358795 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-envoy\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.359032 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.359011 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-podinfo\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.365703 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.365675 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-token\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.365792 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.365739 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nfgs\" (UniqueName: \"kubernetes.io/projected/0457ed66-7e63-4bcf-a489-16c40ebc25a1-kube-api-access-6nfgs\") pod \"e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.443580 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.443490 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:02.584741 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.584706 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn"] Jul 28 20:17:02.587552 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:17:02.587521 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0457ed66_7e63_4bcf_a489_16c40ebc25a1.slice/crio-15dc1d10591ba827b1c13e95bd11cc04491f06df7a295c784345d385a39b45cf WatchSource:0}: Error finding container 15dc1d10591ba827b1c13e95bd11cc04491f06df7a295c784345d385a39b45cf: Status 404 returned error can't find the container with id 15dc1d10591ba827b1c13e95bd11cc04491f06df7a295c784345d385a39b45cf Jul 28 20:17:02.589788 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.589757 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:17:02.589874 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.589830 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:17:02.596653 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.590250 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:17:02.759410 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.759326 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" event={"ID":"0457ed66-7e63-4bcf-a489-16c40ebc25a1","Type":"ContainerStarted","Data":"f0b8d5e6ee36ca5ff906c8f1f39ff9d77c45252a1a354ecc678912f225efc254"} Jul 28 20:17:02.759549 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.759427 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" event={"ID":"0457ed66-7e63-4bcf-a489-16c40ebc25a1","Type":"ContainerStarted","Data":"15dc1d10591ba827b1c13e95bd11cc04491f06df7a295c784345d385a39b45cf"} Jul 28 20:17:02.781845 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:02.781768 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" podStartSLOduration=0.781746265 podStartE2EDuration="781.746265ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:17:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:17:02.779171839 +0000 UTC m=+2418.486383733" watchObservedRunningTime="2026-07-28 20:17:02.781746265 +0000 UTC m=+2418.488958160" Jul 28 20:17:03.444352 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:03.444317 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:03.449283 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:03.449260 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:03.765515 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:03.765424 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:03.766608 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:03.766587 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:05.625923 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:05.625881 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p"] Jul 28 20:17:05.626529 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:05.626510 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e27a9b9e-9536-428e-a861-3e61c6780049" containerName="istio-proxy" Jul 28 20:17:05.629878 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:05.629857 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" Jul 28 20:17:05.632877 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:05.632859 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-a5ccdf-dockercfg-s5jjf\"" Jul 28 20:17:05.632985 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:05.632860 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-a5ccdf\"" Jul 28 20:17:05.650883 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:05.650849 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p"] Jul 28 20:17:05.691379 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:05.691072 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84d31a81-c119-47af-be30-a11db61e2568-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p\" (UID: \"84d31a81-c119-47af-be30-a11db61e2568\") " pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" Jul 28 20:17:05.691379 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:05.691147 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjc8f\" (UniqueName: \"kubernetes.io/projected/84d31a81-c119-47af-be30-a11db61e2568-kube-api-access-cjc8f\") pod \"payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p\" (UID: \"84d31a81-c119-47af-be30-a11db61e2568\") " pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" Jul 28 20:17:05.792411 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:05.792383 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84d31a81-c119-47af-be30-a11db61e2568-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p\" (UID: \"84d31a81-c119-47af-be30-a11db61e2568\") " pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" Jul 28 20:17:05.792587 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:05.792423 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjc8f\" (UniqueName: \"kubernetes.io/projected/84d31a81-c119-47af-be30-a11db61e2568-kube-api-access-cjc8f\") pod \"payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p\" (UID: \"84d31a81-c119-47af-be30-a11db61e2568\") " pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" Jul 28 20:17:05.793149 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:05.793131 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84d31a81-c119-47af-be30-a11db61e2568-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p\" (UID: \"84d31a81-c119-47af-be30-a11db61e2568\") " pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" Jul 28 20:17:05.807740 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:05.807707 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjc8f\" (UniqueName: \"kubernetes.io/projected/84d31a81-c119-47af-be30-a11db61e2568-kube-api-access-cjc8f\") pod \"payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p\" (UID: \"84d31a81-c119-47af-be30-a11db61e2568\") " pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" Jul 28 20:17:05.941007 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:05.940914 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" Jul 28 20:17:06.085174 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:17:06.085118 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84d31a81_c119_47af_be30_a11db61e2568.slice/crio-a930a8faee3e1878e117d32a9bc0a197ffac930741819d27651fbf102028ba60 WatchSource:0}: Error finding container a930a8faee3e1878e117d32a9bc0a197ffac930741819d27651fbf102028ba60: Status 404 returned error can't find the container with id a930a8faee3e1878e117d32a9bc0a197ffac930741819d27651fbf102028ba60 Jul 28 20:17:06.099797 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:06.099547 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p"] Jul 28 20:17:06.783909 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:06.783874 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" event={"ID":"84d31a81-c119-47af-be30-a11db61e2568","Type":"ContainerStarted","Data":"ce03d0d1d9c4342416ba41ce0ecddbf3e0fdaf89faae1317d64138e77069a8c3"} Jul 28 20:17:06.784384 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:06.783936 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" event={"ID":"84d31a81-c119-47af-be30-a11db61e2568","Type":"ContainerStarted","Data":"a930a8faee3e1878e117d32a9bc0a197ffac930741819d27651fbf102028ba60"} Jul 28 20:17:06.784384 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:06.783964 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" Jul 28 20:17:06.801665 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:06.801606 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" podStartSLOduration=1.801587151 podStartE2EDuration="1.801587151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:17:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:17:06.799425842 +0000 UTC m=+2422.506637736" watchObservedRunningTime="2026-07-28 20:17:06.801587151 +0000 UTC m=+2422.508799043" Jul 28 20:17:17.792031 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:17.791992 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" Jul 28 20:17:19.280462 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:19.280411 2564 patch_prober.go:28] interesting pod/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": read tcp 10.134.0.2:33204->10.134.0.52:15021: read: connection reset by peer" start-of-body= Jul 28 20:17:19.280881 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:19.280487 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" podUID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": read tcp 10.134.0.2:33204->10.134.0.52:15021: read: connection reset by peer" Jul 28 20:17:19.850233 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:19.850199 2564 generic.go:356] "Generic (PLEG): container finished" podID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" containerID="f0b8d5e6ee36ca5ff906c8f1f39ff9d77c45252a1a354ecc678912f225efc254" exitCode=0 Jul 28 20:17:19.850425 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:19.850271 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" event={"ID":"0457ed66-7e63-4bcf-a489-16c40ebc25a1","Type":"ContainerDied","Data":"f0b8d5e6ee36ca5ff906c8f1f39ff9d77c45252a1a354ecc678912f225efc254"} Jul 28 20:17:19.850689 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:19.850673 2564 scope.go:117] "RemoveContainer" containerID="f0b8d5e6ee36ca5ff906c8f1f39ff9d77c45252a1a354ecc678912f225efc254" Jul 28 20:17:19.851669 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:19.851626 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:17:19.851766 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:19.851710 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:17:19.851766 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:19.851740 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:17:20.860713 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:20.860668 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" event={"ID":"0457ed66-7e63-4bcf-a489-16c40ebc25a1","Type":"ContainerStarted","Data":"00d90e6b8ba7f9fb851ca9151af198bc32c0f3bd1e30953d1d761c98a9886ad0"} Jul 28 20:17:21.451908 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:21.451853 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:21.452587 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:21.452124 2564 patch_prober.go:28] interesting pod/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 28 20:17:21.452587 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:21.452276 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" podUID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 28 20:17:22.444467 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:22.443971 2564 patch_prober.go:28] interesting pod/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 28 20:17:22.444467 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:22.444054 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" podUID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 28 20:17:23.444748 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:23.444709 2564 patch_prober.go:28] interesting pod/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 28 20:17:23.445330 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:23.445296 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" podUID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 28 20:17:24.443965 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:24.443923 2564 patch_prober.go:28] interesting pod/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 28 20:17:24.444701 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:24.444256 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" podUID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 28 20:17:25.447804 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:25.447772 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:25.448265 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:25.448138 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:25.448915 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:25.448882 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:17:34.033912 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.033872 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls"] Jul 28 20:17:34.037623 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.037600 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" Jul 28 20:17:34.040527 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.040504 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-bf09e7\"" Jul 28 20:17:34.040679 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.040541 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-bf09e7-dockercfg-pblch\"" Jul 28 20:17:34.046078 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.046053 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls"] Jul 28 20:17:34.089137 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.089097 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/661c55c0-a76f-4c33-a302-15e641e2ec43-plugins-config-volume\") pod \"payload-processing-e2e-api-b-bf09e7-5c564484-cjkls\" (UID: \"661c55c0-a76f-4c33-a302-15e641e2ec43\") " pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" Jul 28 20:17:34.089314 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.089153 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8ss9\" (UniqueName: \"kubernetes.io/projected/661c55c0-a76f-4c33-a302-15e641e2ec43-kube-api-access-l8ss9\") pod \"payload-processing-e2e-api-b-bf09e7-5c564484-cjkls\" (UID: \"661c55c0-a76f-4c33-a302-15e641e2ec43\") " pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" Jul 28 20:17:34.189959 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.189921 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/661c55c0-a76f-4c33-a302-15e641e2ec43-plugins-config-volume\") pod \"payload-processing-e2e-api-b-bf09e7-5c564484-cjkls\" (UID: \"661c55c0-a76f-4c33-a302-15e641e2ec43\") " pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" Jul 28 20:17:34.190157 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.189980 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8ss9\" (UniqueName: \"kubernetes.io/projected/661c55c0-a76f-4c33-a302-15e641e2ec43-kube-api-access-l8ss9\") pod \"payload-processing-e2e-api-b-bf09e7-5c564484-cjkls\" (UID: \"661c55c0-a76f-4c33-a302-15e641e2ec43\") " pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" Jul 28 20:17:34.190649 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.190615 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/661c55c0-a76f-4c33-a302-15e641e2ec43-plugins-config-volume\") pod \"payload-processing-e2e-api-b-bf09e7-5c564484-cjkls\" (UID: \"661c55c0-a76f-4c33-a302-15e641e2ec43\") " pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" Jul 28 20:17:34.198305 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.198279 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8ss9\" (UniqueName: \"kubernetes.io/projected/661c55c0-a76f-4c33-a302-15e641e2ec43-kube-api-access-l8ss9\") pod \"payload-processing-e2e-api-b-bf09e7-5c564484-cjkls\" (UID: \"661c55c0-a76f-4c33-a302-15e641e2ec43\") " pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" Jul 28 20:17:34.349655 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.349600 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" Jul 28 20:17:34.484847 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.484820 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls"] Jul 28 20:17:34.486802 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:17:34.486772 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod661c55c0_a76f_4c33_a302_15e641e2ec43.slice/crio-e780d03a9dc4ad186c4fc92d5a4a46d2b514bae5e077f073aac464ea716e9bd5 WatchSource:0}: Error finding container e780d03a9dc4ad186c4fc92d5a4a46d2b514bae5e077f073aac464ea716e9bd5: Status 404 returned error can't find the container with id e780d03a9dc4ad186c4fc92d5a4a46d2b514bae5e077f073aac464ea716e9bd5 Jul 28 20:17:34.932058 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.931993 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" event={"ID":"661c55c0-a76f-4c33-a302-15e641e2ec43","Type":"ContainerStarted","Data":"48ef18b8f06f09472ef83c5ba8e2d657a4e28c29b3287eb85f6c987f57e5d519"} Jul 28 20:17:34.932265 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.932078 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" Jul 28 20:17:34.932265 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.932097 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" event={"ID":"661c55c0-a76f-4c33-a302-15e641e2ec43","Type":"ContainerStarted","Data":"e780d03a9dc4ad186c4fc92d5a4a46d2b514bae5e077f073aac464ea716e9bd5"} Jul 28 20:17:34.949280 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:34.949230 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" podStartSLOduration=0.949215072 podStartE2EDuration="949.215072ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:17:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:17:34.948577567 +0000 UTC m=+2450.655789463" watchObservedRunningTime="2026-07-28 20:17:34.949215072 +0000 UTC m=+2450.656426964" Jul 28 20:17:45.938873 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:17:45.938838 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" Jul 28 20:18:16.691540 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:16.691499 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p"] Jul 28 20:18:16.692188 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:16.691795 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" podUID="84d31a81-c119-47af-be30-a11db61e2568" containerName="payload-processing" containerID="cri-o://ce03d0d1d9c4342416ba41ce0ecddbf3e0fdaf89faae1317d64138e77069a8c3" gracePeriod=30 Jul 28 20:18:16.947666 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:16.947580 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" Jul 28 20:18:17.010132 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:17.010095 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84d31a81-c119-47af-be30-a11db61e2568-plugins-config-volume\") pod \"84d31a81-c119-47af-be30-a11db61e2568\" (UID: \"84d31a81-c119-47af-be30-a11db61e2568\") " Jul 28 20:18:17.010278 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:17.010189 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjc8f\" (UniqueName: \"kubernetes.io/projected/84d31a81-c119-47af-be30-a11db61e2568-kube-api-access-cjc8f\") pod \"84d31a81-c119-47af-be30-a11db61e2568\" (UID: \"84d31a81-c119-47af-be30-a11db61e2568\") " Jul 28 20:18:17.010551 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:17.010521 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84d31a81-c119-47af-be30-a11db61e2568-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84d31a81-c119-47af-be30-a11db61e2568" (UID: "84d31a81-c119-47af-be30-a11db61e2568"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:18:17.012749 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:17.012722 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84d31a81-c119-47af-be30-a11db61e2568-kube-api-access-cjc8f" (OuterVolumeSpecName: "kube-api-access-cjc8f") pod "84d31a81-c119-47af-be30-a11db61e2568" (UID: "84d31a81-c119-47af-be30-a11db61e2568"). InnerVolumeSpecName "kube-api-access-cjc8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:18:17.111763 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:17.111691 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84d31a81-c119-47af-be30-a11db61e2568-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:18:17.111941 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:17.111756 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjc8f\" (UniqueName: \"kubernetes.io/projected/84d31a81-c119-47af-be30-a11db61e2568-kube-api-access-cjc8f\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:18:17.143325 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:17.143288 2564 generic.go:356] "Generic (PLEG): container finished" podID="84d31a81-c119-47af-be30-a11db61e2568" containerID="ce03d0d1d9c4342416ba41ce0ecddbf3e0fdaf89faae1317d64138e77069a8c3" exitCode=0 Jul 28 20:18:17.143487 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:17.143351 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" Jul 28 20:18:17.143487 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:17.143352 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" event={"ID":"84d31a81-c119-47af-be30-a11db61e2568","Type":"ContainerDied","Data":"ce03d0d1d9c4342416ba41ce0ecddbf3e0fdaf89faae1317d64138e77069a8c3"} Jul 28 20:18:17.143574 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:17.143495 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p" event={"ID":"84d31a81-c119-47af-be30-a11db61e2568","Type":"ContainerDied","Data":"a930a8faee3e1878e117d32a9bc0a197ffac930741819d27651fbf102028ba60"} Jul 28 20:18:17.143574 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:17.143524 2564 scope.go:117] "RemoveContainer" containerID="ce03d0d1d9c4342416ba41ce0ecddbf3e0fdaf89faae1317d64138e77069a8c3" Jul 28 20:18:17.152611 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:17.152585 2564 scope.go:117] "RemoveContainer" containerID="ce03d0d1d9c4342416ba41ce0ecddbf3e0fdaf89faae1317d64138e77069a8c3" Jul 28 20:18:17.152920 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:18:17.152896 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce03d0d1d9c4342416ba41ce0ecddbf3e0fdaf89faae1317d64138e77069a8c3\": container with ID starting with ce03d0d1d9c4342416ba41ce0ecddbf3e0fdaf89faae1317d64138e77069a8c3 not found: ID does not exist" containerID="ce03d0d1d9c4342416ba41ce0ecddbf3e0fdaf89faae1317d64138e77069a8c3" Jul 28 20:18:17.153007 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:17.152927 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce03d0d1d9c4342416ba41ce0ecddbf3e0fdaf89faae1317d64138e77069a8c3"} err="failed to get container status \"ce03d0d1d9c4342416ba41ce0ecddbf3e0fdaf89faae1317d64138e77069a8c3\": rpc error: code = NotFound desc = could not find container \"ce03d0d1d9c4342416ba41ce0ecddbf3e0fdaf89faae1317d64138e77069a8c3\": container with ID starting with ce03d0d1d9c4342416ba41ce0ecddbf3e0fdaf89faae1317d64138e77069a8c3 not found: ID does not exist" Jul 28 20:18:17.165391 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:17.165366 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p"] Jul 28 20:18:17.171056 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:17.171027 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a5ccdf-79f7ff56cb-xgq4p"] Jul 28 20:18:18.918859 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:18.918824 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84d31a81-c119-47af-be30-a11db61e2568" path="/var/lib/kubelet/pods/84d31a81-c119-47af-be30-a11db61e2568/volumes" Jul 28 20:18:23.327577 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:23.327547 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn"] Jul 28 20:18:23.327965 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:23.327784 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" podUID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" containerName="istio-proxy" containerID="cri-o://00d90e6b8ba7f9fb851ca9151af198bc32c0f3bd1e30953d1d761c98a9886ad0" gracePeriod=30 Jul 28 20:18:25.450723 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:25.450683 2564 patch_prober.go:28] interesting pod/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 28 20:18:25.451132 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:25.450740 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" podUID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 20:18:28.582525 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.582493 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:18:28.724745 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.724631 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-workload-socket\") pod \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " Jul 28 20:18:28.724745 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.724715 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istiod-ca-cert\") pod \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " Jul 28 20:18:28.724745 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.724743 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-credential-socket\") pod \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " Jul 28 20:18:28.725028 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.724764 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nfgs\" (UniqueName: \"kubernetes.io/projected/0457ed66-7e63-4bcf-a489-16c40ebc25a1-kube-api-access-6nfgs\") pod \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " Jul 28 20:18:28.725028 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.724800 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-token\") pod \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " Jul 28 20:18:28.725028 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.724847 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-data\") pod \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " Jul 28 20:18:28.725028 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.724875 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-podinfo\") pod \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " Jul 28 20:18:28.725028 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.724873 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0457ed66-7e63-4bcf-a489-16c40ebc25a1" (UID: "0457ed66-7e63-4bcf-a489-16c40ebc25a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:18:28.725028 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.724929 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-workload-certs\") pod \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " Jul 28 20:18:28.725028 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.724981 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-envoy\") pod \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\" (UID: \"0457ed66-7e63-4bcf-a489-16c40ebc25a1\") " Jul 28 20:18:28.725387 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.725256 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0457ed66-7e63-4bcf-a489-16c40ebc25a1" (UID: "0457ed66-7e63-4bcf-a489-16c40ebc25a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:18:28.725387 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.725258 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0457ed66-7e63-4bcf-a489-16c40ebc25a1" (UID: "0457ed66-7e63-4bcf-a489-16c40ebc25a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:18:28.725387 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.725342 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-workload-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:18:28.725387 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.725354 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0457ed66-7e63-4bcf-a489-16c40ebc25a1" (UID: "0457ed66-7e63-4bcf-a489-16c40ebc25a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:18:28.725906 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.725874 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-data" (OuterVolumeSpecName: "istio-data") pod "0457ed66-7e63-4bcf-a489-16c40ebc25a1" (UID: "0457ed66-7e63-4bcf-a489-16c40ebc25a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:18:28.727104 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.727081 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-token" (OuterVolumeSpecName: "istio-token") pod "0457ed66-7e63-4bcf-a489-16c40ebc25a1" (UID: "0457ed66-7e63-4bcf-a489-16c40ebc25a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:18:28.727199 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.727081 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0457ed66-7e63-4bcf-a489-16c40ebc25a1-kube-api-access-6nfgs" (OuterVolumeSpecName: "kube-api-access-6nfgs") pod "0457ed66-7e63-4bcf-a489-16c40ebc25a1" (UID: "0457ed66-7e63-4bcf-a489-16c40ebc25a1"). InnerVolumeSpecName "kube-api-access-6nfgs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:18:28.727199 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.727130 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0457ed66-7e63-4bcf-a489-16c40ebc25a1" (UID: "0457ed66-7e63-4bcf-a489-16c40ebc25a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:18:28.727368 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.727340 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0457ed66-7e63-4bcf-a489-16c40ebc25a1" (UID: "0457ed66-7e63-4bcf-a489-16c40ebc25a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 20:18:28.826484 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.826454 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istiod-ca-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:18:28.826484 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.826480 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-credential-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:18:28.826484 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.826490 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nfgs\" (UniqueName: \"kubernetes.io/projected/0457ed66-7e63-4bcf-a489-16c40ebc25a1-kube-api-access-6nfgs\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:18:28.826727 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.826498 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-token\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:18:28.826727 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.826506 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-data\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:18:28.826727 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.826515 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-podinfo\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:18:28.826727 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.826523 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-workload-certs\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:18:28.826727 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:28.826532 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0457ed66-7e63-4bcf-a489-16c40ebc25a1-istio-envoy\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:18:29.204583 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:29.204547 2564 generic.go:356] "Generic (PLEG): container finished" podID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" containerID="00d90e6b8ba7f9fb851ca9151af198bc32c0f3bd1e30953d1d761c98a9886ad0" exitCode=0 Jul 28 20:18:29.204787 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:29.204600 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" Jul 28 20:18:29.204787 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:29.204596 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" event={"ID":"0457ed66-7e63-4bcf-a489-16c40ebc25a1","Type":"ContainerDied","Data":"00d90e6b8ba7f9fb851ca9151af198bc32c0f3bd1e30953d1d761c98a9886ad0"} Jul 28 20:18:29.204787 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:29.204679 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn" event={"ID":"0457ed66-7e63-4bcf-a489-16c40ebc25a1","Type":"ContainerDied","Data":"15dc1d10591ba827b1c13e95bd11cc04491f06df7a295c784345d385a39b45cf"} Jul 28 20:18:29.204787 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:29.204704 2564 scope.go:117] "RemoveContainer" containerID="00d90e6b8ba7f9fb851ca9151af198bc32c0f3bd1e30953d1d761c98a9886ad0" Jul 28 20:18:29.215207 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:29.215177 2564 scope.go:117] "RemoveContainer" containerID="f0b8d5e6ee36ca5ff906c8f1f39ff9d77c45252a1a354ecc678912f225efc254" Jul 28 20:18:29.223194 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:29.223178 2564 scope.go:117] "RemoveContainer" containerID="00d90e6b8ba7f9fb851ca9151af198bc32c0f3bd1e30953d1d761c98a9886ad0" Jul 28 20:18:29.223444 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:18:29.223427 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00d90e6b8ba7f9fb851ca9151af198bc32c0f3bd1e30953d1d761c98a9886ad0\": container with ID starting with 00d90e6b8ba7f9fb851ca9151af198bc32c0f3bd1e30953d1d761c98a9886ad0 not found: ID does not exist" containerID="00d90e6b8ba7f9fb851ca9151af198bc32c0f3bd1e30953d1d761c98a9886ad0" Jul 28 20:18:29.223515 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:29.223457 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00d90e6b8ba7f9fb851ca9151af198bc32c0f3bd1e30953d1d761c98a9886ad0"} err="failed to get container status \"00d90e6b8ba7f9fb851ca9151af198bc32c0f3bd1e30953d1d761c98a9886ad0\": rpc error: code = NotFound desc = could not find container \"00d90e6b8ba7f9fb851ca9151af198bc32c0f3bd1e30953d1d761c98a9886ad0\": container with ID starting with 00d90e6b8ba7f9fb851ca9151af198bc32c0f3bd1e30953d1d761c98a9886ad0 not found: ID does not exist" Jul 28 20:18:29.223515 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:29.223479 2564 scope.go:117] "RemoveContainer" containerID="f0b8d5e6ee36ca5ff906c8f1f39ff9d77c45252a1a354ecc678912f225efc254" Jul 28 20:18:29.223740 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:18:29.223723 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0b8d5e6ee36ca5ff906c8f1f39ff9d77c45252a1a354ecc678912f225efc254\": container with ID starting with f0b8d5e6ee36ca5ff906c8f1f39ff9d77c45252a1a354ecc678912f225efc254 not found: ID does not exist" containerID="f0b8d5e6ee36ca5ff906c8f1f39ff9d77c45252a1a354ecc678912f225efc254" Jul 28 20:18:29.223788 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:29.223746 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0b8d5e6ee36ca5ff906c8f1f39ff9d77c45252a1a354ecc678912f225efc254"} err="failed to get container status \"f0b8d5e6ee36ca5ff906c8f1f39ff9d77c45252a1a354ecc678912f225efc254\": rpc error: code = NotFound desc = could not find container \"f0b8d5e6ee36ca5ff906c8f1f39ff9d77c45252a1a354ecc678912f225efc254\": container with ID starting with f0b8d5e6ee36ca5ff906c8f1f39ff9d77c45252a1a354ecc678912f225efc254 not found: ID does not exist" Jul 28 20:18:29.226582 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:29.226557 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn"] Jul 28 20:18:29.230977 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:29.230957 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a5ccdf-openshift-default-79874dddd-nxjzn"] Jul 28 20:18:30.919352 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:30.919317 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" path="/var/lib/kubelet/pods/0457ed66-7e63-4bcf-a489-16c40ebc25a1/volumes" Jul 28 20:18:39.135434 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:39.135398 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls"] Jul 28 20:18:39.135855 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:39.135690 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" podUID="661c55c0-a76f-4c33-a302-15e641e2ec43" containerName="payload-processing" containerID="cri-o://48ef18b8f06f09472ef83c5ba8e2d657a4e28c29b3287eb85f6c987f57e5d519" gracePeriod=30 Jul 28 20:18:39.254942 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:39.254909 2564 generic.go:356] "Generic (PLEG): container finished" podID="661c55c0-a76f-4c33-a302-15e641e2ec43" containerID="48ef18b8f06f09472ef83c5ba8e2d657a4e28c29b3287eb85f6c987f57e5d519" exitCode=0 Jul 28 20:18:39.255080 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:39.254951 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" event={"ID":"661c55c0-a76f-4c33-a302-15e641e2ec43","Type":"ContainerDied","Data":"48ef18b8f06f09472ef83c5ba8e2d657a4e28c29b3287eb85f6c987f57e5d519"} Jul 28 20:18:39.386372 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:39.386315 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" Jul 28 20:18:39.431426 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:39.431398 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8ss9\" (UniqueName: \"kubernetes.io/projected/661c55c0-a76f-4c33-a302-15e641e2ec43-kube-api-access-l8ss9\") pod \"661c55c0-a76f-4c33-a302-15e641e2ec43\" (UID: \"661c55c0-a76f-4c33-a302-15e641e2ec43\") " Jul 28 20:18:39.431580 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:39.431451 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/661c55c0-a76f-4c33-a302-15e641e2ec43-plugins-config-volume\") pod \"661c55c0-a76f-4c33-a302-15e641e2ec43\" (UID: \"661c55c0-a76f-4c33-a302-15e641e2ec43\") " Jul 28 20:18:39.431861 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:39.431825 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/661c55c0-a76f-4c33-a302-15e641e2ec43-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "661c55c0-a76f-4c33-a302-15e641e2ec43" (UID: "661c55c0-a76f-4c33-a302-15e641e2ec43"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:18:39.433513 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:39.433482 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/661c55c0-a76f-4c33-a302-15e641e2ec43-kube-api-access-l8ss9" (OuterVolumeSpecName: "kube-api-access-l8ss9") pod "661c55c0-a76f-4c33-a302-15e641e2ec43" (UID: "661c55c0-a76f-4c33-a302-15e641e2ec43"). InnerVolumeSpecName "kube-api-access-l8ss9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:18:39.532716 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:39.532668 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8ss9\" (UniqueName: \"kubernetes.io/projected/661c55c0-a76f-4c33-a302-15e641e2ec43-kube-api-access-l8ss9\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:18:39.532716 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:39.532711 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/661c55c0-a76f-4c33-a302-15e641e2ec43-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:18:40.260973 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:40.260929 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" event={"ID":"661c55c0-a76f-4c33-a302-15e641e2ec43","Type":"ContainerDied","Data":"e780d03a9dc4ad186c4fc92d5a4a46d2b514bae5e077f073aac464ea716e9bd5"} Jul 28 20:18:40.260973 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:40.260959 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls" Jul 28 20:18:40.261461 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:40.261010 2564 scope.go:117] "RemoveContainer" containerID="48ef18b8f06f09472ef83c5ba8e2d657a4e28c29b3287eb85f6c987f57e5d519" Jul 28 20:18:40.283363 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:40.283335 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls"] Jul 28 20:18:40.288340 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:40.288316 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bf09e7-5c564484-cjkls"] Jul 28 20:18:40.919094 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:40.919056 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="661c55c0-a76f-4c33-a302-15e641e2ec43" path="/var/lib/kubelet/pods/661c55c0-a76f-4c33-a302-15e641e2ec43/volumes" Jul 28 20:18:46.821391 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.821350 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg"] Jul 28 20:18:46.821843 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.821756 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84d31a81-c119-47af-be30-a11db61e2568" containerName="payload-processing" Jul 28 20:18:46.821843 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.821769 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="84d31a81-c119-47af-be30-a11db61e2568" containerName="payload-processing" Jul 28 20:18:46.821843 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.821780 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" containerName="istio-proxy" Jul 28 20:18:46.821843 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.821786 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" containerName="istio-proxy" Jul 28 20:18:46.821843 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.821810 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" containerName="istio-proxy" Jul 28 20:18:46.821843 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.821817 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" containerName="istio-proxy" Jul 28 20:18:46.821843 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.821834 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="661c55c0-a76f-4c33-a302-15e641e2ec43" containerName="payload-processing" Jul 28 20:18:46.821843 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.821839 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="661c55c0-a76f-4c33-a302-15e641e2ec43" containerName="payload-processing" Jul 28 20:18:46.822131 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.821893 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="84d31a81-c119-47af-be30-a11db61e2568" containerName="payload-processing" Jul 28 20:18:46.822131 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.821902 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" containerName="istio-proxy" Jul 28 20:18:46.822131 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.821908 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="661c55c0-a76f-4c33-a302-15e641e2ec43" containerName="payload-processing" Jul 28 20:18:46.826212 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.826195 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:46.829623 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.829605 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a1e1ef-openshift-default-dockercfg-l52jp\"" Jul 28 20:18:46.852584 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.852557 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg"] Jul 28 20:18:46.900564 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.900522 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-envoy\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:46.900762 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.900581 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istiod-ca-cert\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:46.900762 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.900601 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-token\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:46.900762 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.900625 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-podinfo\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:46.900762 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.900712 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-credential-socket\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:46.900762 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.900747 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-workload-certs\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:46.900936 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.900794 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-data\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:46.900936 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.900818 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-workload-socket\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:46.900936 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:46.900841 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vdmv\" (UniqueName: \"kubernetes.io/projected/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-kube-api-access-7vdmv\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.002087 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.002039 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-credential-socket\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.002087 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.002087 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-workload-certs\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.002309 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.002131 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-data\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.002309 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.002149 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-workload-socket\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.002309 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.002208 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vdmv\" (UniqueName: \"kubernetes.io/projected/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-kube-api-access-7vdmv\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.002309 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.002266 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-envoy\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.002499 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.002397 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istiod-ca-cert\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.002499 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.002427 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-token\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.002499 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.002459 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-credential-socket\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.002499 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.002476 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-podinfo\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.002499 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.002484 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-workload-socket\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.002795 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.002529 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-data\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.002834 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.002805 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-workload-certs\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.003078 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.003054 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istiod-ca-cert\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.004479 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.004458 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-envoy\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.004826 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.004810 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-podinfo\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.012755 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.012709 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-token\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.012863 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.012748 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vdmv\" (UniqueName: \"kubernetes.io/projected/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-kube-api-access-7vdmv\") pod \"e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.137116 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.137020 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:47.273696 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.273669 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg"] Jul 28 20:18:47.275770 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:18:47.275739 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34e0c56e_8389_4d1d_8ec9_01fcc08d359a.slice/crio-323ca15493306b5727894f8c1a74f516e9c378c540f6bf98caece64364106b4a WatchSource:0}: Error finding container 323ca15493306b5727894f8c1a74f516e9c378c540f6bf98caece64364106b4a: Status 404 returned error can't find the container with id 323ca15493306b5727894f8c1a74f516e9c378c540f6bf98caece64364106b4a Jul 28 20:18:47.278001 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.277962 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:18:47.278066 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.278050 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:18:47.278111 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.278095 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:18:47.303958 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:47.303921 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" event={"ID":"34e0c56e-8389-4d1d-8ec9-01fcc08d359a","Type":"ContainerStarted","Data":"323ca15493306b5727894f8c1a74f516e9c378c540f6bf98caece64364106b4a"} Jul 28 20:18:48.310428 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:48.310389 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" event={"ID":"34e0c56e-8389-4d1d-8ec9-01fcc08d359a","Type":"ContainerStarted","Data":"a63564ba2fc64ac8a418feb42fb0f6d59a2aa54fdae388c20cd2ec27ceaf8e16"} Jul 28 20:18:48.332978 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:48.332923 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" podStartSLOduration=2.332906875 podStartE2EDuration="2.332906875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:18:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:18:48.33076301 +0000 UTC m=+2524.037974905" watchObservedRunningTime="2026-07-28 20:18:48.332906875 +0000 UTC m=+2524.040118768" Jul 28 20:18:49.137910 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:49.137871 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:49.142993 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:49.142966 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:49.314943 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:49.314913 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:49.315830 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:49.315811 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:18:51.132519 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:51.132486 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69"] Jul 28 20:18:51.132952 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:51.132939 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0457ed66-7e63-4bcf-a489-16c40ebc25a1" containerName="istio-proxy" Jul 28 20:18:51.136183 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:51.136164 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" Jul 28 20:18:51.138818 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:51.138794 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-a1e1ef\"" Jul 28 20:18:51.138931 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:51.138802 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-a1e1ef-dockercfg-l7plf\"" Jul 28 20:18:51.148321 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:51.148291 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69"] Jul 28 20:18:51.244879 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:51.244845 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69\" (UID: \"1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" Jul 28 20:18:51.245070 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:51.245038 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8s89\" (UniqueName: \"kubernetes.io/projected/1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8-kube-api-access-j8s89\") pod \"payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69\" (UID: \"1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" Jul 28 20:18:51.346568 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:51.346531 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8s89\" (UniqueName: \"kubernetes.io/projected/1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8-kube-api-access-j8s89\") pod \"payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69\" (UID: \"1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" Jul 28 20:18:51.346796 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:51.346589 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69\" (UID: \"1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" Jul 28 20:18:51.347378 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:51.347359 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69\" (UID: \"1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" Jul 28 20:18:51.355514 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:51.355485 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8s89\" (UniqueName: \"kubernetes.io/projected/1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8-kube-api-access-j8s89\") pod \"payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69\" (UID: \"1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" Jul 28 20:18:51.447814 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:51.447721 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" Jul 28 20:18:51.583565 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:51.583530 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69"] Jul 28 20:18:51.587116 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:18:51.587087 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e37a3ae_49fa_49e6_87a5_4f4e11d8bde8.slice/crio-0875f432600a4cd9b23d77f416ab2a5842d01710a3591e76be442492672c3803 WatchSource:0}: Error finding container 0875f432600a4cd9b23d77f416ab2a5842d01710a3591e76be442492672c3803: Status 404 returned error can't find the container with id 0875f432600a4cd9b23d77f416ab2a5842d01710a3591e76be442492672c3803 Jul 28 20:18:52.329698 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:52.329664 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" event={"ID":"1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8","Type":"ContainerStarted","Data":"6ec09326ba3ba3910de2645e488658e561efe0cf3d180f592c07919d694a08e9"} Jul 28 20:18:52.330121 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:52.329842 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" Jul 28 20:18:52.330121 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:52.329864 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" event={"ID":"1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8","Type":"ContainerStarted","Data":"0875f432600a4cd9b23d77f416ab2a5842d01710a3591e76be442492672c3803"} Jul 28 20:18:52.346927 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:18:52.346866 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" podStartSLOduration=1.346845686 podStartE2EDuration="1.346845686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:18:52.346833169 +0000 UTC m=+2528.054045064" watchObservedRunningTime="2026-07-28 20:18:52.346845686 +0000 UTC m=+2528.054057583" Jul 28 20:19:03.337265 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:03.337232 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" Jul 28 20:19:03.394705 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:03.394675 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" event={"ID":"34e0c56e-8389-4d1d-8ec9-01fcc08d359a","Type":"ContainerDied","Data":"a63564ba2fc64ac8a418feb42fb0f6d59a2aa54fdae388c20cd2ec27ceaf8e16"} Jul 28 20:19:03.394984 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:03.394707 2564 generic.go:356] "Generic (PLEG): container finished" podID="34e0c56e-8389-4d1d-8ec9-01fcc08d359a" containerID="a63564ba2fc64ac8a418feb42fb0f6d59a2aa54fdae388c20cd2ec27ceaf8e16" exitCode=0 Jul 28 20:19:03.395129 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:03.395113 2564 scope.go:117] "RemoveContainer" containerID="a63564ba2fc64ac8a418feb42fb0f6d59a2aa54fdae388c20cd2ec27ceaf8e16" Jul 28 20:19:03.396242 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:03.396210 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:19:03.396313 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:03.396297 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:19:03.396363 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:03.396347 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:19:04.315221 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:04.315191 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:19:04.402327 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:04.402294 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" event={"ID":"34e0c56e-8389-4d1d-8ec9-01fcc08d359a","Type":"ContainerStarted","Data":"c7a34fe0d044bc19104c0e88c7a21395e2f3676464806a22e4bf304e91819004"} Jul 28 20:19:05.137285 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:05.137248 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:19:05.138711 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:05.138680 2564 patch_prober.go:28] interesting pod/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 28 20:19:05.139008 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:05.138956 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" podUID="34e0c56e-8389-4d1d-8ec9-01fcc08d359a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 28 20:19:06.138205 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:06.138153 2564 patch_prober.go:28] interesting pod/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 28 20:19:06.138895 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:06.138859 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" podUID="34e0c56e-8389-4d1d-8ec9-01fcc08d359a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 28 20:19:07.137354 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:07.137321 2564 patch_prober.go:28] interesting pod/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 28 20:19:07.137551 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:07.137386 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" podUID="34e0c56e-8389-4d1d-8ec9-01fcc08d359a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 28 20:19:08.137416 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:08.137372 2564 patch_prober.go:28] interesting pod/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 28 20:19:08.137982 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:08.137435 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" podUID="34e0c56e-8389-4d1d-8ec9-01fcc08d359a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 28 20:19:09.142189 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:09.142153 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:19:09.142659 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:09.142417 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:19:09.143159 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:09.143139 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:19:19.987544 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:19.987511 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k"] Jul 28 20:19:19.990349 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:19.990331 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" Jul 28 20:19:19.993754 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:19.993732 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-47f87c-dockercfg-796f7\"" Jul 28 20:19:19.993899 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:19.993750 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-47f87c\"" Jul 28 20:19:20.023978 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.023933 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k"] Jul 28 20:19:20.034195 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.034150 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs4mh\" (UniqueName: \"kubernetes.io/projected/1277a003-f9ae-4de5-bd73-da20b2b97327-kube-api-access-hs4mh\") pod \"payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k\" (UID: \"1277a003-f9ae-4de5-bd73-da20b2b97327\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" Jul 28 20:19:20.034394 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.034208 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1277a003-f9ae-4de5-bd73-da20b2b97327-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k\" (UID: \"1277a003-f9ae-4de5-bd73-da20b2b97327\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" Jul 28 20:19:20.055251 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.055214 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds"] Jul 28 20:19:20.057992 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.057967 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" Jul 28 20:19:20.074335 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.074303 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds"] Jul 28 20:19:20.135363 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.135324 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56bac63b-37be-45e3-b6e8-034d0f2443ef-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds\" (UID: \"56bac63b-37be-45e3-b6e8-034d0f2443ef\") " pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" Jul 28 20:19:20.135594 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.135454 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs4mh\" (UniqueName: \"kubernetes.io/projected/1277a003-f9ae-4de5-bd73-da20b2b97327-kube-api-access-hs4mh\") pod \"payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k\" (UID: \"1277a003-f9ae-4de5-bd73-da20b2b97327\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" Jul 28 20:19:20.135594 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.135504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1277a003-f9ae-4de5-bd73-da20b2b97327-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k\" (UID: \"1277a003-f9ae-4de5-bd73-da20b2b97327\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" Jul 28 20:19:20.135748 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.135590 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7kvz\" (UniqueName: \"kubernetes.io/projected/56bac63b-37be-45e3-b6e8-034d0f2443ef-kube-api-access-c7kvz\") pod \"payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds\" (UID: \"56bac63b-37be-45e3-b6e8-034d0f2443ef\") " pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" Jul 28 20:19:20.136338 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.136319 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1277a003-f9ae-4de5-bd73-da20b2b97327-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k\" (UID: \"1277a003-f9ae-4de5-bd73-da20b2b97327\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" Jul 28 20:19:20.145018 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.144993 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs4mh\" (UniqueName: \"kubernetes.io/projected/1277a003-f9ae-4de5-bd73-da20b2b97327-kube-api-access-hs4mh\") pod \"payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k\" (UID: \"1277a003-f9ae-4de5-bd73-da20b2b97327\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" Jul 28 20:19:20.236963 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.236925 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7kvz\" (UniqueName: \"kubernetes.io/projected/56bac63b-37be-45e3-b6e8-034d0f2443ef-kube-api-access-c7kvz\") pod \"payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds\" (UID: \"56bac63b-37be-45e3-b6e8-034d0f2443ef\") " pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" Jul 28 20:19:20.236963 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.236967 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56bac63b-37be-45e3-b6e8-034d0f2443ef-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds\" (UID: \"56bac63b-37be-45e3-b6e8-034d0f2443ef\") " pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" Jul 28 20:19:20.237685 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.237608 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56bac63b-37be-45e3-b6e8-034d0f2443ef-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds\" (UID: \"56bac63b-37be-45e3-b6e8-034d0f2443ef\") " pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" Jul 28 20:19:20.247740 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.247716 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7kvz\" (UniqueName: \"kubernetes.io/projected/56bac63b-37be-45e3-b6e8-034d0f2443ef-kube-api-access-c7kvz\") pod \"payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds\" (UID: \"56bac63b-37be-45e3-b6e8-034d0f2443ef\") " pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" Jul 28 20:19:20.300584 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.300546 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" Jul 28 20:19:20.367992 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.367956 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" Jul 28 20:19:20.444716 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.444690 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k"] Jul 28 20:19:20.446684 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:19:20.446657 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1277a003_f9ae_4de5_bd73_da20b2b97327.slice/crio-0b5944254348ed5147b78c7883cff4804dab10b58883b8d407d95c337b90a452 WatchSource:0}: Error finding container 0b5944254348ed5147b78c7883cff4804dab10b58883b8d407d95c337b90a452: Status 404 returned error can't find the container with id 0b5944254348ed5147b78c7883cff4804dab10b58883b8d407d95c337b90a452 Jul 28 20:19:20.492899 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.492863 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" event={"ID":"1277a003-f9ae-4de5-bd73-da20b2b97327","Type":"ContainerStarted","Data":"0b5944254348ed5147b78c7883cff4804dab10b58883b8d407d95c337b90a452"} Jul 28 20:19:20.526328 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:20.526295 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds"] Jul 28 20:19:20.529823 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:19:20.529789 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56bac63b_37be_45e3_b6e8_034d0f2443ef.slice/crio-b620502d2215b996dfd1fe9c1c112562b5267f8d707408810c420836cef5f5ee WatchSource:0}: Error finding container b620502d2215b996dfd1fe9c1c112562b5267f8d707408810c420836cef5f5ee: Status 404 returned error can't find the container with id b620502d2215b996dfd1fe9c1c112562b5267f8d707408810c420836cef5f5ee Jul 28 20:19:21.499585 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:21.499546 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" event={"ID":"1277a003-f9ae-4de5-bd73-da20b2b97327","Type":"ContainerStarted","Data":"c6462d5ac46cfd38713d24ec4caa62c580965fc6705244909a048542ff2142e9"} Jul 28 20:19:21.500070 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:21.499647 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" Jul 28 20:19:21.501114 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:21.501089 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" event={"ID":"56bac63b-37be-45e3-b6e8-034d0f2443ef","Type":"ContainerStarted","Data":"ace87957632bd9ef71d1c4a060dbc087b2b4fd384b599dceaf8d5e6f63b5db84"} Jul 28 20:19:21.501236 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:21.501137 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" event={"ID":"56bac63b-37be-45e3-b6e8-034d0f2443ef","Type":"ContainerStarted","Data":"b620502d2215b996dfd1fe9c1c112562b5267f8d707408810c420836cef5f5ee"} Jul 28 20:19:21.501236 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:21.501164 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" Jul 28 20:19:21.522664 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:21.522589 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" podStartSLOduration=2.522572374 podStartE2EDuration="2.522572374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:19:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:19:21.520553068 +0000 UTC m=+2557.227764962" watchObservedRunningTime="2026-07-28 20:19:21.522572374 +0000 UTC m=+2557.229784267" Jul 28 20:19:21.540360 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:21.540297 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" podStartSLOduration=1.540284836 podStartE2EDuration="1.540284836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:19:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:19:21.538747346 +0000 UTC m=+2557.245959248" watchObservedRunningTime="2026-07-28 20:19:21.540284836 +0000 UTC m=+2557.247496730" Jul 28 20:19:32.508077 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:32.508047 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" Jul 28 20:19:32.508467 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:19:32.508113 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" Jul 28 20:28:48.539708 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.539675 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw"] Jul 28 20:28:48.542145 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.542129 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.545592 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.545574 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-aa68dd-openshift-default-dockercfg-j65dj\"" Jul 28 20:28:48.566696 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.566672 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw"] Jul 28 20:28:48.704450 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.704414 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-workload-socket\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.704684 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.704470 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-token\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.704684 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.704551 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-workload-certs\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.704684 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.704592 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-envoy\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.704684 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.704629 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-credential-socket\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.704911 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.704690 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-podinfo\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.704911 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.704721 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-data\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.704911 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.704754 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4hn5\" (UniqueName: \"kubernetes.io/projected/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-kube-api-access-p4hn5\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.704911 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.704800 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istiod-ca-cert\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.806246 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.806215 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-token\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.806437 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.806270 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-workload-certs\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.806437 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.806302 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-envoy\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.806437 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.806327 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-credential-socket\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.806437 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.806347 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-podinfo\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.806437 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.806370 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-data\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.806437 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.806400 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4hn5\" (UniqueName: \"kubernetes.io/projected/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-kube-api-access-p4hn5\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.806797 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.806450 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istiod-ca-cert\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.806797 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.806678 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-workload-socket\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.806910 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.806849 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-credential-socket\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.806975 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.806932 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-data\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.806975 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.806963 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-workload-certs\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.807085 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.807055 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-workload-socket\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.807171 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.807148 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istiod-ca-cert\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.808611 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.808589 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-envoy\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.809234 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.809209 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-podinfo\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.826256 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.826234 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-token\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.826547 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.826525 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4hn5\" (UniqueName: \"kubernetes.io/projected/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-kube-api-access-p4hn5\") pod \"e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:48.852307 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:48.852272 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:49.008820 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:49.008793 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw"] Jul 28 20:28:49.010913 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:28:49.010878 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7400e1b_ce61_4dc8_b0c0_9199f9c15395.slice/crio-9736d07bbdda1a893b80b878e2a82a34b5b77bf13903b1ce07fe8427ec06eb03 WatchSource:0}: Error finding container 9736d07bbdda1a893b80b878e2a82a34b5b77bf13903b1ce07fe8427ec06eb03: Status 404 returned error can't find the container with id 9736d07bbdda1a893b80b878e2a82a34b5b77bf13903b1ce07fe8427ec06eb03 Jul 28 20:28:49.012985 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:49.012953 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:28:49.013092 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:49.013029 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:28:49.013156 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:49.013096 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:28:49.386566 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:49.386464 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" event={"ID":"b7400e1b-ce61-4dc8-b0c0-9199f9c15395","Type":"ContainerStarted","Data":"3c6cf2378152403b3488e6396e3e565aae0d675458970f6c56a252a463466c82"} Jul 28 20:28:49.386566 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:49.386527 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" event={"ID":"b7400e1b-ce61-4dc8-b0c0-9199f9c15395","Type":"ContainerStarted","Data":"9736d07bbdda1a893b80b878e2a82a34b5b77bf13903b1ce07fe8427ec06eb03"} Jul 28 20:28:49.423013 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:49.422964 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" podStartSLOduration=1.422947797 podStartE2EDuration="1.422947797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:28:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:28:49.422109113 +0000 UTC m=+3125.129321022" watchObservedRunningTime="2026-07-28 20:28:49.422947797 +0000 UTC m=+3125.130159689" Jul 28 20:28:49.853275 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:49.853244 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:50.858407 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:50.858375 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:51.400200 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:51.400165 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:51.404120 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:51.404091 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:28:52.796558 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.796524 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n"] Jul 28 20:28:52.799263 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.799247 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" Jul 28 20:28:52.804038 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.804023 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-aa68dd\"" Jul 28 20:28:52.814780 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.814749 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n"] Jul 28 20:28:52.842900 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.842862 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79rpd\" (UniqueName: \"kubernetes.io/projected/4400cb29-8868-4e2f-9871-b82bf4e85dce-kube-api-access-79rpd\") pod \"payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n\" (UID: \"4400cb29-8868-4e2f-9871-b82bf4e85dce\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" Jul 28 20:28:52.843078 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.842945 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4400cb29-8868-4e2f-9871-b82bf4e85dce-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n\" (UID: \"4400cb29-8868-4e2f-9871-b82bf4e85dce\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" Jul 28 20:28:52.847631 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.847601 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq"] Jul 28 20:28:52.851450 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.851427 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" Jul 28 20:28:52.861652 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.861614 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-aa68dd-dockercfg-tclhg\"" Jul 28 20:28:52.868836 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.868811 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq"] Jul 28 20:28:52.945120 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.945058 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79rpd\" (UniqueName: \"kubernetes.io/projected/4400cb29-8868-4e2f-9871-b82bf4e85dce-kube-api-access-79rpd\") pod \"payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n\" (UID: \"4400cb29-8868-4e2f-9871-b82bf4e85dce\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" Jul 28 20:28:52.945319 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.945163 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4400cb29-8868-4e2f-9871-b82bf4e85dce-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n\" (UID: \"4400cb29-8868-4e2f-9871-b82bf4e85dce\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" Jul 28 20:28:52.945319 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.945210 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7466d8d-0b76-48f9-bf86-931aa5ba0dd2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq\" (UID: \"c7466d8d-0b76-48f9-bf86-931aa5ba0dd2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" Jul 28 20:28:52.945319 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.945247 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6lw6\" (UniqueName: \"kubernetes.io/projected/c7466d8d-0b76-48f9-bf86-931aa5ba0dd2-kube-api-access-w6lw6\") pod \"payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq\" (UID: \"c7466d8d-0b76-48f9-bf86-931aa5ba0dd2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" Jul 28 20:28:52.946134 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.946108 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4400cb29-8868-4e2f-9871-b82bf4e85dce-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n\" (UID: \"4400cb29-8868-4e2f-9871-b82bf4e85dce\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" Jul 28 20:28:52.957242 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:52.957220 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79rpd\" (UniqueName: \"kubernetes.io/projected/4400cb29-8868-4e2f-9871-b82bf4e85dce-kube-api-access-79rpd\") pod \"payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n\" (UID: \"4400cb29-8868-4e2f-9871-b82bf4e85dce\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" Jul 28 20:28:53.046114 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:53.046081 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7466d8d-0b76-48f9-bf86-931aa5ba0dd2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq\" (UID: \"c7466d8d-0b76-48f9-bf86-931aa5ba0dd2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" Jul 28 20:28:53.046296 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:53.046122 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6lw6\" (UniqueName: \"kubernetes.io/projected/c7466d8d-0b76-48f9-bf86-931aa5ba0dd2-kube-api-access-w6lw6\") pod \"payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq\" (UID: \"c7466d8d-0b76-48f9-bf86-931aa5ba0dd2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" Jul 28 20:28:53.046872 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:53.046815 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7466d8d-0b76-48f9-bf86-931aa5ba0dd2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq\" (UID: \"c7466d8d-0b76-48f9-bf86-931aa5ba0dd2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" Jul 28 20:28:53.073223 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:53.073194 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6lw6\" (UniqueName: \"kubernetes.io/projected/c7466d8d-0b76-48f9-bf86-931aa5ba0dd2-kube-api-access-w6lw6\") pod \"payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq\" (UID: \"c7466d8d-0b76-48f9-bf86-931aa5ba0dd2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" Jul 28 20:28:53.110088 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:53.110057 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" Jul 28 20:28:53.161237 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:53.161210 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" Jul 28 20:28:53.362146 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:53.362121 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq"] Jul 28 20:28:53.364333 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:28:53.364302 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7466d8d_0b76_48f9_bf86_931aa5ba0dd2.slice/crio-9b661307f1084d0857642f1e66915ae210a28f50b96be3de7d71faa488d8a5e0 WatchSource:0}: Error finding container 9b661307f1084d0857642f1e66915ae210a28f50b96be3de7d71faa488d8a5e0: Status 404 returned error can't find the container with id 9b661307f1084d0857642f1e66915ae210a28f50b96be3de7d71faa488d8a5e0 Jul 28 20:28:53.367951 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:53.367929 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n"] Jul 28 20:28:53.368309 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:28:53.368284 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4400cb29_8868_4e2f_9871_b82bf4e85dce.slice/crio-c55ea46d5eeadda944a50a9fdb1a576f42824bb5fa7731d827c3d8c17b16adca WatchSource:0}: Error finding container c55ea46d5eeadda944a50a9fdb1a576f42824bb5fa7731d827c3d8c17b16adca: Status 404 returned error can't find the container with id c55ea46d5eeadda944a50a9fdb1a576f42824bb5fa7731d827c3d8c17b16adca Jul 28 20:28:53.411958 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:53.411930 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" event={"ID":"c7466d8d-0b76-48f9-bf86-931aa5ba0dd2","Type":"ContainerStarted","Data":"9b661307f1084d0857642f1e66915ae210a28f50b96be3de7d71faa488d8a5e0"} Jul 28 20:28:53.413143 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:53.413117 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" event={"ID":"4400cb29-8868-4e2f-9871-b82bf4e85dce","Type":"ContainerStarted","Data":"c55ea46d5eeadda944a50a9fdb1a576f42824bb5fa7731d827c3d8c17b16adca"} Jul 28 20:28:54.419990 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:54.419953 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" event={"ID":"c7466d8d-0b76-48f9-bf86-931aa5ba0dd2","Type":"ContainerStarted","Data":"b702ebbdd641da52b07681cfd67850868e31ca16d51503b22d37891c84ff0744"} Jul 28 20:28:54.420410 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:54.420124 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" Jul 28 20:28:54.421546 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:54.421523 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" event={"ID":"4400cb29-8868-4e2f-9871-b82bf4e85dce","Type":"ContainerStarted","Data":"609229207fc107939153e3a32b6d6f079a516309e567216a4296f161cb5a986d"} Jul 28 20:28:54.421652 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:54.421646 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" Jul 28 20:28:54.458864 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:28:54.458816 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" podStartSLOduration=2.4588011229999998 podStartE2EDuration="2.458801123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:28:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:28:54.457612268 +0000 UTC m=+3130.164824163" watchObservedRunningTime="2026-07-28 20:28:54.458801123 +0000 UTC m=+3130.166013015" Jul 28 20:29:05.428822 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:05.428779 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" Jul 28 20:29:05.429209 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:05.428967 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" Jul 28 20:29:05.448982 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:05.448917 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" podStartSLOduration=13.448896678 podStartE2EDuration="13.448896678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:28:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:28:54.485466412 +0000 UTC m=+3130.192678309" watchObservedRunningTime="2026-07-28 20:29:05.448896678 +0000 UTC m=+3141.156108573" Jul 28 20:29:07.442866 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:07.442772 2564 patch_prober.go:28] interesting pod/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 28 20:29:07.443244 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:07.442871 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" podUID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 28 20:29:08.503721 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:08.503683 2564 generic.go:356] "Generic (PLEG): container finished" podID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerID="3c6cf2378152403b3488e6396e3e565aae0d675458970f6c56a252a463466c82" exitCode=0 Jul 28 20:29:08.504239 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:08.503763 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" event={"ID":"b7400e1b-ce61-4dc8-b0c0-9199f9c15395","Type":"ContainerDied","Data":"3c6cf2378152403b3488e6396e3e565aae0d675458970f6c56a252a463466c82"} Jul 28 20:29:08.504337 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:08.504318 2564 scope.go:117] "RemoveContainer" containerID="3c6cf2378152403b3488e6396e3e565aae0d675458970f6c56a252a463466c82" Jul 28 20:29:08.505359 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:08.505335 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:29:08.505418 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:08.505402 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:29:08.505455 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:08.505431 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:29:09.511740 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:09.511705 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" event={"ID":"b7400e1b-ce61-4dc8-b0c0-9199f9c15395","Type":"ContainerStarted","Data":"2685b95df0bca838fa0cc36f662e5e6e1020554dbc4e2df55e7748036699f66a"} Jul 28 20:29:09.853430 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:09.853384 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:29:09.853934 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:09.853903 2564 patch_prober.go:28] interesting pod/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 28 20:29:09.854059 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:09.853959 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" podUID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 28 20:29:10.852963 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:10.852930 2564 patch_prober.go:28] interesting pod/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 28 20:29:10.853529 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:10.853500 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" podUID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 28 20:29:11.853311 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:11.853265 2564 patch_prober.go:28] interesting pod/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 28 20:29:11.853860 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:11.853343 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" podUID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 28 20:29:12.853170 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:12.853123 2564 patch_prober.go:28] interesting pod/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 28 20:29:12.853351 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:12.853224 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" podUID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 28 20:29:13.853329 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:13.853272 2564 patch_prober.go:28] interesting pod/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 28 20:29:13.853534 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:13.853353 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" podUID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 28 20:29:14.856708 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:14.856674 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:29:14.857118 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:14.856962 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:29:14.857577 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:14.857558 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:29:17.725070 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.725022 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8"] Jul 28 20:29:17.727727 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.727710 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.736919 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.736897 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-4ec840-openshift-default-dockercfg-h85fz\"" Jul 28 20:29:17.798806 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.798769 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-token\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.798974 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.798827 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-workload-socket\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.798974 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.798911 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-workload-certs\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.798974 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.798964 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-envoy\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.799104 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.799032 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-podinfo\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.799104 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.799072 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-data\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.799104 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.799096 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r9mv\" (UniqueName: \"kubernetes.io/projected/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-kube-api-access-6r9mv\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.799321 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.799155 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-credential-socket\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.799321 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.799243 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istiod-ca-cert\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.866091 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.866059 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8"] Jul 28 20:29:17.899988 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.899952 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-data\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.900144 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.899998 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r9mv\" (UniqueName: \"kubernetes.io/projected/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-kube-api-access-6r9mv\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.900144 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.900030 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-credential-socket\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.900144 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.900078 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istiod-ca-cert\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.900144 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.900126 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-token\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.900384 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.900175 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-workload-socket\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.900384 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.900348 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-workload-certs\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.900511 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.900401 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-data\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.900511 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.900402 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-envoy\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.900511 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.900462 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-credential-socket\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.900511 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.900483 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-podinfo\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.900700 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.900529 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-workload-socket\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.900778 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.900754 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-workload-certs\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.900832 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.900805 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istiod-ca-cert\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.902677 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.902652 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-envoy\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.902773 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.902761 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-podinfo\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.937393 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.937360 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r9mv\" (UniqueName: \"kubernetes.io/projected/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-kube-api-access-6r9mv\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:17.947440 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:17.947418 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-token\") pod \"e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:18.038001 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:18.037910 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:18.208027 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:18.207930 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8"] Jul 28 20:29:18.210786 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:29:18.210760 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39d69163_a5aa_4dc2_ab6a_50e14e76e6c1.slice/crio-1781ab97a5d8dc3fd361c40d1e1c00847a8e333560bc90c78c0c6bf0892a4d1b WatchSource:0}: Error finding container 1781ab97a5d8dc3fd361c40d1e1c00847a8e333560bc90c78c0c6bf0892a4d1b: Status 404 returned error can't find the container with id 1781ab97a5d8dc3fd361c40d1e1c00847a8e333560bc90c78c0c6bf0892a4d1b Jul 28 20:29:18.212929 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:18.212896 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:29:18.213010 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:18.212965 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:29:18.213010 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:18.212994 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:29:18.562676 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:18.562628 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" event={"ID":"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1","Type":"ContainerStarted","Data":"02e681f0e77f1f31e4a002cd2b60645cc0beb82e6464baff1fd971f73b4b5f53"} Jul 28 20:29:18.562877 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:18.562692 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" event={"ID":"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1","Type":"ContainerStarted","Data":"1781ab97a5d8dc3fd361c40d1e1c00847a8e333560bc90c78c0c6bf0892a4d1b"} Jul 28 20:29:18.587963 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:18.587903 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" podStartSLOduration=1.587889202 podStartE2EDuration="1.587889202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:29:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:29:18.586847546 +0000 UTC m=+3154.294059461" watchObservedRunningTime="2026-07-28 20:29:18.587889202 +0000 UTC m=+3154.295101095" Jul 28 20:29:19.038919 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:19.038836 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:19.043918 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:19.043897 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:19.568529 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:19.568495 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:19.569403 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:19.569386 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:38.691117 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:38.691078 2564 generic.go:356] "Generic (PLEG): container finished" podID="39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" containerID="02e681f0e77f1f31e4a002cd2b60645cc0beb82e6464baff1fd971f73b4b5f53" exitCode=0 Jul 28 20:29:38.691117 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:38.691120 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" event={"ID":"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1","Type":"ContainerDied","Data":"02e681f0e77f1f31e4a002cd2b60645cc0beb82e6464baff1fd971f73b4b5f53"} Jul 28 20:29:38.691583 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:38.691487 2564 scope.go:117] "RemoveContainer" containerID="02e681f0e77f1f31e4a002cd2b60645cc0beb82e6464baff1fd971f73b4b5f53" Jul 28 20:29:38.692547 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:38.692522 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:29:38.692607 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:38.692596 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:29:38.692664 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:38.692625 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 28 20:29:39.699406 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:39.699374 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" event={"ID":"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1","Type":"ContainerStarted","Data":"c5e04064a3b40cf324d5321480eabde73128c9e77129851072a03bd9b3565b4f"} Jul 28 20:29:40.038414 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:40.038311 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:40.038594 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:40.038438 2564 patch_prober.go:28] interesting pod/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 28 20:29:40.038594 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:40.038481 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" podUID="39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 28 20:29:41.038970 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:41.038928 2564 patch_prober.go:28] interesting pod/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 28 20:29:41.039605 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:41.039568 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" podUID="39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 28 20:29:42.039251 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:42.038787 2564 patch_prober.go:28] interesting pod/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 28 20:29:42.039251 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:42.038855 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" podUID="39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 28 20:29:43.038612 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:43.038568 2564 patch_prober.go:28] interesting pod/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 28 20:29:43.038865 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:43.038672 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" podUID="39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 28 20:29:44.042104 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:44.042075 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:44.042519 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:44.042330 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:29:44.043077 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:29:44.043059 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:30:54.851056 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:30:54.851022 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq_c7466d8d-0b76-48f9-bf86-931aa5ba0dd2/payload-processing/0.log" Jul 28 20:30:59.914428 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:30:59.914378 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx"] Jul 28 20:30:59.918688 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:30:59.918658 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" Jul 28 20:30:59.922420 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:30:59.922395 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-e5feff\"" Jul 28 20:30:59.922664 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:30:59.922513 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-e5feff-dockercfg-zwfjh\"" Jul 28 20:30:59.939444 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:30:59.939397 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx"] Jul 28 20:31:00.050597 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:00.050559 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d795a9-d7ff-4db5-863c-e57ce8312c90-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx\" (UID: \"c4d795a9-d7ff-4db5-863c-e57ce8312c90\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" Jul 28 20:31:00.050827 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:00.050617 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nkph\" (UniqueName: \"kubernetes.io/projected/c4d795a9-d7ff-4db5-863c-e57ce8312c90-kube-api-access-6nkph\") pod \"payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx\" (UID: \"c4d795a9-d7ff-4db5-863c-e57ce8312c90\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" Jul 28 20:31:00.152165 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:00.152131 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nkph\" (UniqueName: \"kubernetes.io/projected/c4d795a9-d7ff-4db5-863c-e57ce8312c90-kube-api-access-6nkph\") pod \"payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx\" (UID: \"c4d795a9-d7ff-4db5-863c-e57ce8312c90\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" Jul 28 20:31:00.152372 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:00.152271 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d795a9-d7ff-4db5-863c-e57ce8312c90-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx\" (UID: \"c4d795a9-d7ff-4db5-863c-e57ce8312c90\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" Jul 28 20:31:00.152992 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:00.152973 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d795a9-d7ff-4db5-863c-e57ce8312c90-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx\" (UID: \"c4d795a9-d7ff-4db5-863c-e57ce8312c90\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" Jul 28 20:31:00.163796 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:00.163772 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nkph\" (UniqueName: \"kubernetes.io/projected/c4d795a9-d7ff-4db5-863c-e57ce8312c90-kube-api-access-6nkph\") pod \"payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx\" (UID: \"c4d795a9-d7ff-4db5-863c-e57ce8312c90\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" Jul 28 20:31:00.235018 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:00.234914 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" Jul 28 20:31:00.379449 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:00.379423 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx"] Jul 28 20:31:00.382490 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:31:00.382464 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d795a9_d7ff_4db5_863c_e57ce8312c90.slice/crio-29a6380050169a2a77489d713fe84f84bbfb6af0e3e983c2082b670af33d00f8 WatchSource:0}: Error finding container 29a6380050169a2a77489d713fe84f84bbfb6af0e3e983c2082b670af33d00f8: Status 404 returned error can't find the container with id 29a6380050169a2a77489d713fe84f84bbfb6af0e3e983c2082b670af33d00f8 Jul 28 20:31:01.135292 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:01.135255 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" event={"ID":"c4d795a9-d7ff-4db5-863c-e57ce8312c90","Type":"ContainerStarted","Data":"272eb6ba43c7d2f044be41db0628d823e88bd814d6a8bcdc2d426f3c314f1f53"} Jul 28 20:31:01.135800 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:01.135314 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" event={"ID":"c4d795a9-d7ff-4db5-863c-e57ce8312c90","Type":"ContainerStarted","Data":"29a6380050169a2a77489d713fe84f84bbfb6af0e3e983c2082b670af33d00f8"} Jul 28 20:31:01.135800 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:01.135340 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" Jul 28 20:31:01.174755 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:01.174706 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" podStartSLOduration=2.17469227 podStartE2EDuration="2.17469227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:30:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:31:01.173200379 +0000 UTC m=+3256.880412273" watchObservedRunningTime="2026-07-28 20:31:01.17469227 +0000 UTC m=+3256.881904162" Jul 28 20:31:12.143252 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:12.143222 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" Jul 28 20:31:29.276368 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:29.276335 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx"] Jul 28 20:31:29.276989 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:29.276552 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" podUID="c4d795a9-d7ff-4db5-863c-e57ce8312c90" containerName="payload-processing" containerID="cri-o://272eb6ba43c7d2f044be41db0628d823e88bd814d6a8bcdc2d426f3c314f1f53" gracePeriod=30 Jul 28 20:31:29.543014 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:29.542991 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" Jul 28 20:31:29.658832 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:29.658799 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nkph\" (UniqueName: \"kubernetes.io/projected/c4d795a9-d7ff-4db5-863c-e57ce8312c90-kube-api-access-6nkph\") pod \"c4d795a9-d7ff-4db5-863c-e57ce8312c90\" (UID: \"c4d795a9-d7ff-4db5-863c-e57ce8312c90\") " Jul 28 20:31:29.659028 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:29.658858 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d795a9-d7ff-4db5-863c-e57ce8312c90-plugins-config-volume\") pod \"c4d795a9-d7ff-4db5-863c-e57ce8312c90\" (UID: \"c4d795a9-d7ff-4db5-863c-e57ce8312c90\") " Jul 28 20:31:29.659277 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:29.659252 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4d795a9-d7ff-4db5-863c-e57ce8312c90-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c4d795a9-d7ff-4db5-863c-e57ce8312c90" (UID: "c4d795a9-d7ff-4db5-863c-e57ce8312c90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:31:29.660938 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:29.660911 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4d795a9-d7ff-4db5-863c-e57ce8312c90-kube-api-access-6nkph" (OuterVolumeSpecName: "kube-api-access-6nkph") pod "c4d795a9-d7ff-4db5-863c-e57ce8312c90" (UID: "c4d795a9-d7ff-4db5-863c-e57ce8312c90"). InnerVolumeSpecName "kube-api-access-6nkph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:31:29.759866 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:29.759832 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d795a9-d7ff-4db5-863c-e57ce8312c90-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:31:29.759866 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:29.759860 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nkph\" (UniqueName: \"kubernetes.io/projected/c4d795a9-d7ff-4db5-863c-e57ce8312c90-kube-api-access-6nkph\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:31:30.288212 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:30.288176 2564 generic.go:356] "Generic (PLEG): container finished" podID="c4d795a9-d7ff-4db5-863c-e57ce8312c90" containerID="272eb6ba43c7d2f044be41db0628d823e88bd814d6a8bcdc2d426f3c314f1f53" exitCode=0 Jul 28 20:31:30.288607 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:30.288227 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" Jul 28 20:31:30.288607 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:30.288260 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" event={"ID":"c4d795a9-d7ff-4db5-863c-e57ce8312c90","Type":"ContainerDied","Data":"272eb6ba43c7d2f044be41db0628d823e88bd814d6a8bcdc2d426f3c314f1f53"} Jul 28 20:31:30.288607 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:30.288316 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx" event={"ID":"c4d795a9-d7ff-4db5-863c-e57ce8312c90","Type":"ContainerDied","Data":"29a6380050169a2a77489d713fe84f84bbfb6af0e3e983c2082b670af33d00f8"} Jul 28 20:31:30.288607 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:30.288339 2564 scope.go:117] "RemoveContainer" containerID="272eb6ba43c7d2f044be41db0628d823e88bd814d6a8bcdc2d426f3c314f1f53" Jul 28 20:31:30.297709 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:30.297691 2564 scope.go:117] "RemoveContainer" containerID="272eb6ba43c7d2f044be41db0628d823e88bd814d6a8bcdc2d426f3c314f1f53" Jul 28 20:31:30.298006 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:31:30.297989 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"272eb6ba43c7d2f044be41db0628d823e88bd814d6a8bcdc2d426f3c314f1f53\": container with ID starting with 272eb6ba43c7d2f044be41db0628d823e88bd814d6a8bcdc2d426f3c314f1f53 not found: ID does not exist" containerID="272eb6ba43c7d2f044be41db0628d823e88bd814d6a8bcdc2d426f3c314f1f53" Jul 28 20:31:30.298055 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:30.298017 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"272eb6ba43c7d2f044be41db0628d823e88bd814d6a8bcdc2d426f3c314f1f53"} err="failed to get container status \"272eb6ba43c7d2f044be41db0628d823e88bd814d6a8bcdc2d426f3c314f1f53\": rpc error: code = NotFound desc = could not find container \"272eb6ba43c7d2f044be41db0628d823e88bd814d6a8bcdc2d426f3c314f1f53\": container with ID starting with 272eb6ba43c7d2f044be41db0628d823e88bd814d6a8bcdc2d426f3c314f1f53 not found: ID does not exist" Jul 28 20:31:30.316159 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:30.316135 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx"] Jul 28 20:31:30.318960 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:30.318939 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e5feff-56b56c7d5d-zbqjx"] Jul 28 20:31:30.919211 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:30.919173 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4d795a9-d7ff-4db5-863c-e57ce8312c90" path="/var/lib/kubelet/pods/c4d795a9-d7ff-4db5-863c-e57ce8312c90/volumes" Jul 28 20:31:57.232666 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.232601 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq"] Jul 28 20:31:57.233191 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.232874 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" podUID="c7466d8d-0b76-48f9-bf86-931aa5ba0dd2" containerName="payload-processing" containerID="cri-o://b702ebbdd641da52b07681cfd67850868e31ca16d51503b22d37891c84ff0744" gracePeriod=30 Jul 28 20:31:57.245843 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.245813 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n"] Jul 28 20:31:57.246563 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.246047 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" podUID="4400cb29-8868-4e2f-9871-b82bf4e85dce" containerName="payload-pre-processing" containerID="cri-o://609229207fc107939153e3a32b6d6f079a516309e567216a4296f161cb5a986d" gracePeriod=30 Jul 28 20:31:57.442496 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.442445 2564 generic.go:356] "Generic (PLEG): container finished" podID="c7466d8d-0b76-48f9-bf86-931aa5ba0dd2" containerID="b702ebbdd641da52b07681cfd67850868e31ca16d51503b22d37891c84ff0744" exitCode=0 Jul 28 20:31:57.442716 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.442556 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" event={"ID":"c7466d8d-0b76-48f9-bf86-931aa5ba0dd2","Type":"ContainerDied","Data":"b702ebbdd641da52b07681cfd67850868e31ca16d51503b22d37891c84ff0744"} Jul 28 20:31:57.444814 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.444783 2564 generic.go:356] "Generic (PLEG): container finished" podID="4400cb29-8868-4e2f-9871-b82bf4e85dce" containerID="609229207fc107939153e3a32b6d6f079a516309e567216a4296f161cb5a986d" exitCode=0 Jul 28 20:31:57.444968 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.444829 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" event={"ID":"4400cb29-8868-4e2f-9871-b82bf4e85dce","Type":"ContainerDied","Data":"609229207fc107939153e3a32b6d6f079a516309e567216a4296f161cb5a986d"} Jul 28 20:31:57.559096 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.559066 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" Jul 28 20:31:57.562193 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.562169 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" Jul 28 20:31:57.637115 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.637090 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4400cb29-8868-4e2f-9871-b82bf4e85dce-plugins-config-volume\") pod \"4400cb29-8868-4e2f-9871-b82bf4e85dce\" (UID: \"4400cb29-8868-4e2f-9871-b82bf4e85dce\") " Jul 28 20:31:57.637309 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.637158 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79rpd\" (UniqueName: \"kubernetes.io/projected/4400cb29-8868-4e2f-9871-b82bf4e85dce-kube-api-access-79rpd\") pod \"4400cb29-8868-4e2f-9871-b82bf4e85dce\" (UID: \"4400cb29-8868-4e2f-9871-b82bf4e85dce\") " Jul 28 20:31:57.637309 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.637243 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7466d8d-0b76-48f9-bf86-931aa5ba0dd2-plugins-config-volume\") pod \"c7466d8d-0b76-48f9-bf86-931aa5ba0dd2\" (UID: \"c7466d8d-0b76-48f9-bf86-931aa5ba0dd2\") " Jul 28 20:31:57.637432 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.637323 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6lw6\" (UniqueName: \"kubernetes.io/projected/c7466d8d-0b76-48f9-bf86-931aa5ba0dd2-kube-api-access-w6lw6\") pod \"c7466d8d-0b76-48f9-bf86-931aa5ba0dd2\" (UID: \"c7466d8d-0b76-48f9-bf86-931aa5ba0dd2\") " Jul 28 20:31:57.637571 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.637541 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4400cb29-8868-4e2f-9871-b82bf4e85dce-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4400cb29-8868-4e2f-9871-b82bf4e85dce" (UID: "4400cb29-8868-4e2f-9871-b82bf4e85dce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:31:57.637704 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.637673 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7466d8d-0b76-48f9-bf86-931aa5ba0dd2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c7466d8d-0b76-48f9-bf86-931aa5ba0dd2" (UID: "c7466d8d-0b76-48f9-bf86-931aa5ba0dd2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:31:57.637835 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.637722 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4400cb29-8868-4e2f-9871-b82bf4e85dce-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:31:57.639382 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.639354 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7466d8d-0b76-48f9-bf86-931aa5ba0dd2-kube-api-access-w6lw6" (OuterVolumeSpecName: "kube-api-access-w6lw6") pod "c7466d8d-0b76-48f9-bf86-931aa5ba0dd2" (UID: "c7466d8d-0b76-48f9-bf86-931aa5ba0dd2"). InnerVolumeSpecName "kube-api-access-w6lw6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:31:57.639463 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.639413 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4400cb29-8868-4e2f-9871-b82bf4e85dce-kube-api-access-79rpd" (OuterVolumeSpecName: "kube-api-access-79rpd") pod "4400cb29-8868-4e2f-9871-b82bf4e85dce" (UID: "4400cb29-8868-4e2f-9871-b82bf4e85dce"). InnerVolumeSpecName "kube-api-access-79rpd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:31:57.738281 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.738251 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6lw6\" (UniqueName: \"kubernetes.io/projected/c7466d8d-0b76-48f9-bf86-931aa5ba0dd2-kube-api-access-w6lw6\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:31:57.738281 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.738277 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79rpd\" (UniqueName: \"kubernetes.io/projected/4400cb29-8868-4e2f-9871-b82bf4e85dce-kube-api-access-79rpd\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:31:57.738281 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:57.738286 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7466d8d-0b76-48f9-bf86-931aa5ba0dd2-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:31:58.452118 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:58.452082 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" Jul 28 20:31:58.452615 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:58.452087 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq" event={"ID":"c7466d8d-0b76-48f9-bf86-931aa5ba0dd2","Type":"ContainerDied","Data":"9b661307f1084d0857642f1e66915ae210a28f50b96be3de7d71faa488d8a5e0"} Jul 28 20:31:58.452615 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:58.452214 2564 scope.go:117] "RemoveContainer" containerID="b702ebbdd641da52b07681cfd67850868e31ca16d51503b22d37891c84ff0744" Jul 28 20:31:58.453649 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:58.453600 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" event={"ID":"4400cb29-8868-4e2f-9871-b82bf4e85dce","Type":"ContainerDied","Data":"c55ea46d5eeadda944a50a9fdb1a576f42824bb5fa7731d827c3d8c17b16adca"} Jul 28 20:31:58.453649 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:58.453627 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n" Jul 28 20:31:58.463048 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:58.463031 2564 scope.go:117] "RemoveContainer" containerID="609229207fc107939153e3a32b6d6f079a516309e567216a4296f161cb5a986d" Jul 28 20:31:58.483072 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:58.483050 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq"] Jul 28 20:31:58.488794 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:58.488770 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aa68dd-6dcb88965d-q65qq"] Jul 28 20:31:58.502726 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:58.502704 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n"] Jul 28 20:31:58.507752 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:58.507732 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aa68dd-d5f8bc4bd-v9v6n"] Jul 28 20:31:58.918926 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:58.918895 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4400cb29-8868-4e2f-9871-b82bf4e85dce" path="/var/lib/kubelet/pods/4400cb29-8868-4e2f-9871-b82bf4e85dce/volumes" Jul 28 20:31:58.919270 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:31:58.919257 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7466d8d-0b76-48f9-bf86-931aa5ba0dd2" path="/var/lib/kubelet/pods/c7466d8d-0b76-48f9-bf86-931aa5ba0dd2/volumes" Jul 28 20:32:42.056380 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:42.056333 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw"] Jul 28 20:32:42.057330 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:42.056605 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" podUID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerName="istio-proxy" containerID="cri-o://2685b95df0bca838fa0cc36f662e5e6e1020554dbc4e2df55e7748036699f66a" gracePeriod=30 Jul 28 20:32:44.859751 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:44.859710 2564 patch_prober.go:28] interesting pod/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 28 20:32:44.860193 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:44.859792 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" podUID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 20:32:47.311070 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.311046 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:32:47.425965 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.425872 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-data\") pod \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " Jul 28 20:32:47.425965 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.425911 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-podinfo\") pod \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " Jul 28 20:32:47.425965 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.425960 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4hn5\" (UniqueName: \"kubernetes.io/projected/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-kube-api-access-p4hn5\") pod \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " Jul 28 20:32:47.426292 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.426022 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-token\") pod \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " Jul 28 20:32:47.426292 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.426077 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-credential-socket\") pod \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " Jul 28 20:32:47.426292 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.426128 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istiod-ca-cert\") pod \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " Jul 28 20:32:47.426292 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.426158 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-workload-certs\") pod \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " Jul 28 20:32:47.426292 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.426211 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-workload-socket\") pod \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " Jul 28 20:32:47.426292 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.426247 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-envoy\") pod \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\" (UID: \"b7400e1b-ce61-4dc8-b0c0-9199f9c15395\") " Jul 28 20:32:47.426624 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.426409 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b7400e1b-ce61-4dc8-b0c0-9199f9c15395" (UID: "b7400e1b-ce61-4dc8-b0c0-9199f9c15395"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:32:47.426624 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.426537 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b7400e1b-ce61-4dc8-b0c0-9199f9c15395" (UID: "b7400e1b-ce61-4dc8-b0c0-9199f9c15395"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:32:47.426624 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.426538 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b7400e1b-ce61-4dc8-b0c0-9199f9c15395" (UID: "b7400e1b-ce61-4dc8-b0c0-9199f9c15395"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:32:47.426624 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.426589 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-credential-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:32:47.426624 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.426618 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b7400e1b-ce61-4dc8-b0c0-9199f9c15395" (UID: "b7400e1b-ce61-4dc8-b0c0-9199f9c15395"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:32:47.426921 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.426742 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-data" (OuterVolumeSpecName: "istio-data") pod "b7400e1b-ce61-4dc8-b0c0-9199f9c15395" (UID: "b7400e1b-ce61-4dc8-b0c0-9199f9c15395"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:32:47.428328 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.428303 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-kube-api-access-p4hn5" (OuterVolumeSpecName: "kube-api-access-p4hn5") pod "b7400e1b-ce61-4dc8-b0c0-9199f9c15395" (UID: "b7400e1b-ce61-4dc8-b0c0-9199f9c15395"). InnerVolumeSpecName "kube-api-access-p4hn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:32:47.428328 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.428317 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b7400e1b-ce61-4dc8-b0c0-9199f9c15395" (UID: "b7400e1b-ce61-4dc8-b0c0-9199f9c15395"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 20:32:47.428535 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.428439 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-token" (OuterVolumeSpecName: "istio-token") pod "b7400e1b-ce61-4dc8-b0c0-9199f9c15395" (UID: "b7400e1b-ce61-4dc8-b0c0-9199f9c15395"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:32:47.428535 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.428481 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b7400e1b-ce61-4dc8-b0c0-9199f9c15395" (UID: "b7400e1b-ce61-4dc8-b0c0-9199f9c15395"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:32:47.527874 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.527825 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istiod-ca-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:32:47.527874 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.527864 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-workload-certs\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:32:47.527874 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.527876 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-workload-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:32:47.528116 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.527888 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-envoy\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:32:47.528116 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.527911 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-data\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:32:47.528116 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.527922 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-podinfo\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:32:47.528116 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.527935 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4hn5\" (UniqueName: \"kubernetes.io/projected/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-kube-api-access-p4hn5\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:32:47.528116 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.527948 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7400e1b-ce61-4dc8-b0c0-9199f9c15395-istio-token\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:32:47.708385 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.708293 2564 generic.go:356] "Generic (PLEG): container finished" podID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerID="2685b95df0bca838fa0cc36f662e5e6e1020554dbc4e2df55e7748036699f66a" exitCode=0 Jul 28 20:32:47.708385 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.708342 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" Jul 28 20:32:47.708385 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.708373 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" event={"ID":"b7400e1b-ce61-4dc8-b0c0-9199f9c15395","Type":"ContainerDied","Data":"2685b95df0bca838fa0cc36f662e5e6e1020554dbc4e2df55e7748036699f66a"} Jul 28 20:32:47.708670 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.708433 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw" event={"ID":"b7400e1b-ce61-4dc8-b0c0-9199f9c15395","Type":"ContainerDied","Data":"9736d07bbdda1a893b80b878e2a82a34b5b77bf13903b1ce07fe8427ec06eb03"} Jul 28 20:32:47.708670 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.708464 2564 scope.go:117] "RemoveContainer" containerID="2685b95df0bca838fa0cc36f662e5e6e1020554dbc4e2df55e7748036699f66a" Jul 28 20:32:47.718605 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.718587 2564 scope.go:117] "RemoveContainer" containerID="3c6cf2378152403b3488e6396e3e565aae0d675458970f6c56a252a463466c82" Jul 28 20:32:47.726971 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.726954 2564 scope.go:117] "RemoveContainer" containerID="2685b95df0bca838fa0cc36f662e5e6e1020554dbc4e2df55e7748036699f66a" Jul 28 20:32:47.727237 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:32:47.727214 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2685b95df0bca838fa0cc36f662e5e6e1020554dbc4e2df55e7748036699f66a\": container with ID starting with 2685b95df0bca838fa0cc36f662e5e6e1020554dbc4e2df55e7748036699f66a not found: ID does not exist" containerID="2685b95df0bca838fa0cc36f662e5e6e1020554dbc4e2df55e7748036699f66a" Jul 28 20:32:47.727308 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.727249 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2685b95df0bca838fa0cc36f662e5e6e1020554dbc4e2df55e7748036699f66a"} err="failed to get container status \"2685b95df0bca838fa0cc36f662e5e6e1020554dbc4e2df55e7748036699f66a\": rpc error: code = NotFound desc = could not find container \"2685b95df0bca838fa0cc36f662e5e6e1020554dbc4e2df55e7748036699f66a\": container with ID starting with 2685b95df0bca838fa0cc36f662e5e6e1020554dbc4e2df55e7748036699f66a not found: ID does not exist" Jul 28 20:32:47.727308 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.727271 2564 scope.go:117] "RemoveContainer" containerID="3c6cf2378152403b3488e6396e3e565aae0d675458970f6c56a252a463466c82" Jul 28 20:32:47.727502 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:32:47.727484 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c6cf2378152403b3488e6396e3e565aae0d675458970f6c56a252a463466c82\": container with ID starting with 3c6cf2378152403b3488e6396e3e565aae0d675458970f6c56a252a463466c82 not found: ID does not exist" containerID="3c6cf2378152403b3488e6396e3e565aae0d675458970f6c56a252a463466c82" Jul 28 20:32:47.727543 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.727510 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c6cf2378152403b3488e6396e3e565aae0d675458970f6c56a252a463466c82"} err="failed to get container status \"3c6cf2378152403b3488e6396e3e565aae0d675458970f6c56a252a463466c82\": rpc error: code = NotFound desc = could not find container \"3c6cf2378152403b3488e6396e3e565aae0d675458970f6c56a252a463466c82\": container with ID starting with 3c6cf2378152403b3488e6396e3e565aae0d675458970f6c56a252a463466c82 not found: ID does not exist" Jul 28 20:32:47.735514 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.735492 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw"] Jul 28 20:32:47.740404 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:47.740381 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-aa68dd-openshift-default-5dfd5546b6-clmvw"] Jul 28 20:32:48.919514 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:32:48.919481 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" path="/var/lib/kubelet/pods/b7400e1b-ce61-4dc8-b0c0-9199f9c15395/volumes" Jul 28 20:33:00.074576 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:00.074542 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8"] Jul 28 20:33:00.075085 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:00.074770 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" podUID="39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" containerName="istio-proxy" containerID="cri-o://c5e04064a3b40cf324d5321480eabde73128c9e77129851072a03bd9b3565b4f" gracePeriod=30 Jul 28 20:33:05.349485 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.349429 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:33:05.416206 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.416174 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-workload-socket\") pod \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " Jul 28 20:33:05.416206 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.416208 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-podinfo\") pod \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " Jul 28 20:33:05.416446 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.416237 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-data\") pod \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " Jul 28 20:33:05.416446 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.416351 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istiod-ca-cert\") pod \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " Jul 28 20:33:05.416446 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.416392 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-token\") pod \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " Jul 28 20:33:05.416446 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.416422 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-credential-socket\") pod \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " Jul 28 20:33:05.416446 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.416427 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" (UID: "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:33:05.416747 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.416521 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6r9mv\" (UniqueName: \"kubernetes.io/projected/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-kube-api-access-6r9mv\") pod \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " Jul 28 20:33:05.416747 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.416556 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-envoy\") pod \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " Jul 28 20:33:05.416747 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.416614 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-workload-certs\") pod \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\" (UID: \"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1\") " Jul 28 20:33:05.416747 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.416662 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" (UID: "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:33:05.416747 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.416720 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" (UID: "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:33:05.416998 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.416981 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-workload-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:05.417050 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.417000 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istiod-ca-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:05.417050 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.417015 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-credential-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:05.417159 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.417074 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" (UID: "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:33:05.417159 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.417127 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-data" (OuterVolumeSpecName: "istio-data") pod "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" (UID: "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:33:05.418602 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.418567 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-token" (OuterVolumeSpecName: "istio-token") pod "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" (UID: "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:33:05.418602 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.418576 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" (UID: "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 20:33:05.418766 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.418714 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" (UID: "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:33:05.418879 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.418856 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-kube-api-access-6r9mv" (OuterVolumeSpecName: "kube-api-access-6r9mv") pod "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" (UID: "39d69163-a5aa-4dc2-ab6a-50e14e76e6c1"). InnerVolumeSpecName "kube-api-access-6r9mv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:33:05.518080 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.518050 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6r9mv\" (UniqueName: \"kubernetes.io/projected/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-kube-api-access-6r9mv\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:05.518080 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.518077 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-envoy\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:05.518080 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.518086 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-workload-certs\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:05.518297 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.518094 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-podinfo\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:05.518297 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.518102 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-data\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:05.518297 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.518116 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1-istio-token\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:05.801827 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.801792 2564 generic.go:356] "Generic (PLEG): container finished" podID="39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" containerID="c5e04064a3b40cf324d5321480eabde73128c9e77129851072a03bd9b3565b4f" exitCode=0 Jul 28 20:33:05.801986 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.801851 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" Jul 28 20:33:05.801986 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.801853 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" event={"ID":"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1","Type":"ContainerDied","Data":"c5e04064a3b40cf324d5321480eabde73128c9e77129851072a03bd9b3565b4f"} Jul 28 20:33:05.801986 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.801909 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8" event={"ID":"39d69163-a5aa-4dc2-ab6a-50e14e76e6c1","Type":"ContainerDied","Data":"1781ab97a5d8dc3fd361c40d1e1c00847a8e333560bc90c78c0c6bf0892a4d1b"} Jul 28 20:33:05.801986 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.801930 2564 scope.go:117] "RemoveContainer" containerID="c5e04064a3b40cf324d5321480eabde73128c9e77129851072a03bd9b3565b4f" Jul 28 20:33:05.818082 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.818063 2564 scope.go:117] "RemoveContainer" containerID="02e681f0e77f1f31e4a002cd2b60645cc0beb82e6464baff1fd971f73b4b5f53" Jul 28 20:33:05.826087 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.826067 2564 scope.go:117] "RemoveContainer" containerID="c5e04064a3b40cf324d5321480eabde73128c9e77129851072a03bd9b3565b4f" Jul 28 20:33:05.826344 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:33:05.826320 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5e04064a3b40cf324d5321480eabde73128c9e77129851072a03bd9b3565b4f\": container with ID starting with c5e04064a3b40cf324d5321480eabde73128c9e77129851072a03bd9b3565b4f not found: ID does not exist" containerID="c5e04064a3b40cf324d5321480eabde73128c9e77129851072a03bd9b3565b4f" Jul 28 20:33:05.826459 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.826349 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5e04064a3b40cf324d5321480eabde73128c9e77129851072a03bd9b3565b4f"} err="failed to get container status \"c5e04064a3b40cf324d5321480eabde73128c9e77129851072a03bd9b3565b4f\": rpc error: code = NotFound desc = could not find container \"c5e04064a3b40cf324d5321480eabde73128c9e77129851072a03bd9b3565b4f\": container with ID starting with c5e04064a3b40cf324d5321480eabde73128c9e77129851072a03bd9b3565b4f not found: ID does not exist" Jul 28 20:33:05.826459 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.826365 2564 scope.go:117] "RemoveContainer" containerID="02e681f0e77f1f31e4a002cd2b60645cc0beb82e6464baff1fd971f73b4b5f53" Jul 28 20:33:05.826659 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:33:05.826614 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02e681f0e77f1f31e4a002cd2b60645cc0beb82e6464baff1fd971f73b4b5f53\": container with ID starting with 02e681f0e77f1f31e4a002cd2b60645cc0beb82e6464baff1fd971f73b4b5f53 not found: ID does not exist" containerID="02e681f0e77f1f31e4a002cd2b60645cc0beb82e6464baff1fd971f73b4b5f53" Jul 28 20:33:05.826752 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.826663 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02e681f0e77f1f31e4a002cd2b60645cc0beb82e6464baff1fd971f73b4b5f53"} err="failed to get container status \"02e681f0e77f1f31e4a002cd2b60645cc0beb82e6464baff1fd971f73b4b5f53\": rpc error: code = NotFound desc = could not find container \"02e681f0e77f1f31e4a002cd2b60645cc0beb82e6464baff1fd971f73b4b5f53\": container with ID starting with 02e681f0e77f1f31e4a002cd2b60645cc0beb82e6464baff1fd971f73b4b5f53 not found: ID does not exist" Jul 28 20:33:05.833665 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.833624 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8"] Jul 28 20:33:05.842358 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:05.842327 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-4ec840-openshift-default-f865484d-ppsx8"] Jul 28 20:33:06.920397 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:06.920367 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" path="/var/lib/kubelet/pods/39d69163-a5aa-4dc2-ab6a-50e14e76e6c1/volumes" Jul 28 20:33:11.200075 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.200045 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69"] Jul 28 20:33:11.200596 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.200247 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" podUID="1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8" containerName="payload-pre-processing" containerID="cri-o://6ec09326ba3ba3910de2645e488658e561efe0cf3d180f592c07919d694a08e9" gracePeriod=30 Jul 28 20:33:11.457525 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.457475 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" Jul 28 20:33:11.574976 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.574944 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8-plugins-config-volume\") pod \"1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8\" (UID: \"1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8\") " Jul 28 20:33:11.575166 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.575018 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8s89\" (UniqueName: \"kubernetes.io/projected/1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8-kube-api-access-j8s89\") pod \"1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8\" (UID: \"1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8\") " Jul 28 20:33:11.575383 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.575356 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8" (UID: "1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:33:11.577058 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.577037 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8-kube-api-access-j8s89" (OuterVolumeSpecName: "kube-api-access-j8s89") pod "1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8" (UID: "1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8"). InnerVolumeSpecName "kube-api-access-j8s89". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:33:11.675791 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.675763 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:11.675791 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.675785 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8s89\" (UniqueName: \"kubernetes.io/projected/1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8-kube-api-access-j8s89\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:11.835277 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.835243 2564 generic.go:356] "Generic (PLEG): container finished" podID="1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8" containerID="6ec09326ba3ba3910de2645e488658e561efe0cf3d180f592c07919d694a08e9" exitCode=0 Jul 28 20:33:11.835430 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.835291 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" Jul 28 20:33:11.835430 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.835298 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" event={"ID":"1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8","Type":"ContainerDied","Data":"6ec09326ba3ba3910de2645e488658e561efe0cf3d180f592c07919d694a08e9"} Jul 28 20:33:11.835430 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.835340 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69" event={"ID":"1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8","Type":"ContainerDied","Data":"0875f432600a4cd9b23d77f416ab2a5842d01710a3591e76be442492672c3803"} Jul 28 20:33:11.835430 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.835360 2564 scope.go:117] "RemoveContainer" containerID="6ec09326ba3ba3910de2645e488658e561efe0cf3d180f592c07919d694a08e9" Jul 28 20:33:11.851386 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.851367 2564 scope.go:117] "RemoveContainer" containerID="6ec09326ba3ba3910de2645e488658e561efe0cf3d180f592c07919d694a08e9" Jul 28 20:33:11.851689 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:33:11.851661 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ec09326ba3ba3910de2645e488658e561efe0cf3d180f592c07919d694a08e9\": container with ID starting with 6ec09326ba3ba3910de2645e488658e561efe0cf3d180f592c07919d694a08e9 not found: ID does not exist" containerID="6ec09326ba3ba3910de2645e488658e561efe0cf3d180f592c07919d694a08e9" Jul 28 20:33:11.851805 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.851699 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ec09326ba3ba3910de2645e488658e561efe0cf3d180f592c07919d694a08e9"} err="failed to get container status \"6ec09326ba3ba3910de2645e488658e561efe0cf3d180f592c07919d694a08e9\": rpc error: code = NotFound desc = could not find container \"6ec09326ba3ba3910de2645e488658e561efe0cf3d180f592c07919d694a08e9\": container with ID starting with 6ec09326ba3ba3910de2645e488658e561efe0cf3d180f592c07919d694a08e9 not found: ID does not exist" Jul 28 20:33:11.861873 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.861841 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69"] Jul 28 20:33:11.863589 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:11.863566 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a1e1ef-5f455bd588-bvd69"] Jul 28 20:33:12.919250 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:12.919213 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8" path="/var/lib/kubelet/pods/1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8/volumes" Jul 28 20:33:18.274510 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:18.274474 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg"] Jul 28 20:33:18.274924 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:18.274747 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" podUID="34e0c56e-8389-4d1d-8ec9-01fcc08d359a" containerName="istio-proxy" containerID="cri-o://c7a34fe0d044bc19104c0e88c7a21395e2f3676464806a22e4bf304e91819004" gracePeriod=30 Jul 28 20:33:23.528706 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.528682 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:33:23.695579 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.695547 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-workload-socket\") pod \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " Jul 28 20:33:23.695804 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.695597 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-data\") pod \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " Jul 28 20:33:23.695804 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.695617 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-workload-certs\") pod \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " Jul 28 20:33:23.695804 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.695665 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-envoy\") pod \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " Jul 28 20:33:23.695804 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.695690 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vdmv\" (UniqueName: \"kubernetes.io/projected/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-kube-api-access-7vdmv\") pod \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " Jul 28 20:33:23.695804 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.695713 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-token\") pod \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " Jul 28 20:33:23.696081 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.695893 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istiod-ca-cert\") pod \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " Jul 28 20:33:23.696081 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.695914 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "34e0c56e-8389-4d1d-8ec9-01fcc08d359a" (UID: "34e0c56e-8389-4d1d-8ec9-01fcc08d359a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:33:23.696081 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.695934 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-credential-socket\") pod \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " Jul 28 20:33:23.696081 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.695969 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-podinfo\") pod \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\" (UID: \"34e0c56e-8389-4d1d-8ec9-01fcc08d359a\") " Jul 28 20:33:23.696081 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.696061 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "34e0c56e-8389-4d1d-8ec9-01fcc08d359a" (UID: "34e0c56e-8389-4d1d-8ec9-01fcc08d359a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:33:23.696334 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.696243 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "34e0c56e-8389-4d1d-8ec9-01fcc08d359a" (UID: "34e0c56e-8389-4d1d-8ec9-01fcc08d359a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:33:23.696334 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.696257 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "34e0c56e-8389-4d1d-8ec9-01fcc08d359a" (UID: "34e0c56e-8389-4d1d-8ec9-01fcc08d359a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:33:23.696434 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.696363 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-workload-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:23.696434 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.696384 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-workload-certs\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:23.696434 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.696398 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istiod-ca-cert\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:23.696434 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.696412 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-credential-socket\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:23.696624 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.696594 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-data" (OuterVolumeSpecName: "istio-data") pod "34e0c56e-8389-4d1d-8ec9-01fcc08d359a" (UID: "34e0c56e-8389-4d1d-8ec9-01fcc08d359a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:33:23.698115 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.698089 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "34e0c56e-8389-4d1d-8ec9-01fcc08d359a" (UID: "34e0c56e-8389-4d1d-8ec9-01fcc08d359a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 20:33:23.698288 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.698269 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "34e0c56e-8389-4d1d-8ec9-01fcc08d359a" (UID: "34e0c56e-8389-4d1d-8ec9-01fcc08d359a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 20:33:23.698374 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.698350 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-token" (OuterVolumeSpecName: "istio-token") pod "34e0c56e-8389-4d1d-8ec9-01fcc08d359a" (UID: "34e0c56e-8389-4d1d-8ec9-01fcc08d359a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:33:23.698428 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.698408 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-kube-api-access-7vdmv" (OuterVolumeSpecName: "kube-api-access-7vdmv") pod "34e0c56e-8389-4d1d-8ec9-01fcc08d359a" (UID: "34e0c56e-8389-4d1d-8ec9-01fcc08d359a"). InnerVolumeSpecName "kube-api-access-7vdmv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:33:23.797902 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.797867 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-podinfo\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:23.797902 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.797896 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-data\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:23.797902 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.797905 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-envoy\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:23.798139 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.797914 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vdmv\" (UniqueName: \"kubernetes.io/projected/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-kube-api-access-7vdmv\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:23.798139 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.797923 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34e0c56e-8389-4d1d-8ec9-01fcc08d359a-istio-token\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:23.897399 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.897367 2564 generic.go:356] "Generic (PLEG): container finished" podID="34e0c56e-8389-4d1d-8ec9-01fcc08d359a" containerID="c7a34fe0d044bc19104c0e88c7a21395e2f3676464806a22e4bf304e91819004" exitCode=0 Jul 28 20:33:23.897567 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.897409 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" Jul 28 20:33:23.897567 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.897455 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" event={"ID":"34e0c56e-8389-4d1d-8ec9-01fcc08d359a","Type":"ContainerDied","Data":"c7a34fe0d044bc19104c0e88c7a21395e2f3676464806a22e4bf304e91819004"} Jul 28 20:33:23.897567 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.897505 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg" event={"ID":"34e0c56e-8389-4d1d-8ec9-01fcc08d359a","Type":"ContainerDied","Data":"323ca15493306b5727894f8c1a74f516e9c378c540f6bf98caece64364106b4a"} Jul 28 20:33:23.897567 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.897527 2564 scope.go:117] "RemoveContainer" containerID="c7a34fe0d044bc19104c0e88c7a21395e2f3676464806a22e4bf304e91819004" Jul 28 20:33:23.907053 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.907032 2564 scope.go:117] "RemoveContainer" containerID="a63564ba2fc64ac8a418feb42fb0f6d59a2aa54fdae388c20cd2ec27ceaf8e16" Jul 28 20:33:23.915562 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.915547 2564 scope.go:117] "RemoveContainer" containerID="c7a34fe0d044bc19104c0e88c7a21395e2f3676464806a22e4bf304e91819004" Jul 28 20:33:23.915848 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:33:23.915831 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7a34fe0d044bc19104c0e88c7a21395e2f3676464806a22e4bf304e91819004\": container with ID starting with c7a34fe0d044bc19104c0e88c7a21395e2f3676464806a22e4bf304e91819004 not found: ID does not exist" containerID="c7a34fe0d044bc19104c0e88c7a21395e2f3676464806a22e4bf304e91819004" Jul 28 20:33:23.915894 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.915855 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7a34fe0d044bc19104c0e88c7a21395e2f3676464806a22e4bf304e91819004"} err="failed to get container status \"c7a34fe0d044bc19104c0e88c7a21395e2f3676464806a22e4bf304e91819004\": rpc error: code = NotFound desc = could not find container \"c7a34fe0d044bc19104c0e88c7a21395e2f3676464806a22e4bf304e91819004\": container with ID starting with c7a34fe0d044bc19104c0e88c7a21395e2f3676464806a22e4bf304e91819004 not found: ID does not exist" Jul 28 20:33:23.915894 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.915871 2564 scope.go:117] "RemoveContainer" containerID="a63564ba2fc64ac8a418feb42fb0f6d59a2aa54fdae388c20cd2ec27ceaf8e16" Jul 28 20:33:23.916099 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:33:23.916082 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a63564ba2fc64ac8a418feb42fb0f6d59a2aa54fdae388c20cd2ec27ceaf8e16\": container with ID starting with a63564ba2fc64ac8a418feb42fb0f6d59a2aa54fdae388c20cd2ec27ceaf8e16 not found: ID does not exist" containerID="a63564ba2fc64ac8a418feb42fb0f6d59a2aa54fdae388c20cd2ec27ceaf8e16" Jul 28 20:33:23.916137 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.916107 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a63564ba2fc64ac8a418feb42fb0f6d59a2aa54fdae388c20cd2ec27ceaf8e16"} err="failed to get container status \"a63564ba2fc64ac8a418feb42fb0f6d59a2aa54fdae388c20cd2ec27ceaf8e16\": rpc error: code = NotFound desc = could not find container \"a63564ba2fc64ac8a418feb42fb0f6d59a2aa54fdae388c20cd2ec27ceaf8e16\": container with ID starting with a63564ba2fc64ac8a418feb42fb0f6d59a2aa54fdae388c20cd2ec27ceaf8e16 not found: ID does not exist" Jul 28 20:33:23.936808 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.936786 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg"] Jul 28 20:33:23.941948 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:23.941925 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a1e1ef-openshift-default-55c5bff676-8sqvg"] Jul 28 20:33:24.918521 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:24.918490 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34e0c56e-8389-4d1d-8ec9-01fcc08d359a" path="/var/lib/kubelet/pods/34e0c56e-8389-4d1d-8ec9-01fcc08d359a/volumes" Jul 28 20:33:29.242133 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.242086 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds"] Jul 28 20:33:29.242615 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.242350 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" podUID="56bac63b-37be-45e3-b6e8-034d0f2443ef" containerName="payload-processing" containerID="cri-o://ace87957632bd9ef71d1c4a060dbc087b2b4fd384b599dceaf8d5e6f63b5db84" gracePeriod=30 Jul 28 20:33:29.249339 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.249314 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k"] Jul 28 20:33:29.249511 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.249492 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" podUID="1277a003-f9ae-4de5-bd73-da20b2b97327" containerName="payload-pre-processing" containerID="cri-o://c6462d5ac46cfd38713d24ec4caa62c580965fc6705244909a048542ff2142e9" gracePeriod=30 Jul 28 20:33:29.500842 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.500780 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" Jul 28 20:33:29.526314 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.526283 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" Jul 28 20:33:29.548981 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.548950 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56bac63b-37be-45e3-b6e8-034d0f2443ef-plugins-config-volume\") pod \"56bac63b-37be-45e3-b6e8-034d0f2443ef\" (UID: \"56bac63b-37be-45e3-b6e8-034d0f2443ef\") " Jul 28 20:33:29.549140 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.549014 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hs4mh\" (UniqueName: \"kubernetes.io/projected/1277a003-f9ae-4de5-bd73-da20b2b97327-kube-api-access-hs4mh\") pod \"1277a003-f9ae-4de5-bd73-da20b2b97327\" (UID: \"1277a003-f9ae-4de5-bd73-da20b2b97327\") " Jul 28 20:33:29.549140 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.549126 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1277a003-f9ae-4de5-bd73-da20b2b97327-plugins-config-volume\") pod \"1277a003-f9ae-4de5-bd73-da20b2b97327\" (UID: \"1277a003-f9ae-4de5-bd73-da20b2b97327\") " Jul 28 20:33:29.549254 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.549180 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7kvz\" (UniqueName: \"kubernetes.io/projected/56bac63b-37be-45e3-b6e8-034d0f2443ef-kube-api-access-c7kvz\") pod \"56bac63b-37be-45e3-b6e8-034d0f2443ef\" (UID: \"56bac63b-37be-45e3-b6e8-034d0f2443ef\") " Jul 28 20:33:29.549469 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.549384 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56bac63b-37be-45e3-b6e8-034d0f2443ef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "56bac63b-37be-45e3-b6e8-034d0f2443ef" (UID: "56bac63b-37be-45e3-b6e8-034d0f2443ef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:33:29.549582 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.549469 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1277a003-f9ae-4de5-bd73-da20b2b97327-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1277a003-f9ae-4de5-bd73-da20b2b97327" (UID: "1277a003-f9ae-4de5-bd73-da20b2b97327"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 20:33:29.549582 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.549504 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56bac63b-37be-45e3-b6e8-034d0f2443ef-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:29.551258 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.551234 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1277a003-f9ae-4de5-bd73-da20b2b97327-kube-api-access-hs4mh" (OuterVolumeSpecName: "kube-api-access-hs4mh") pod "1277a003-f9ae-4de5-bd73-da20b2b97327" (UID: "1277a003-f9ae-4de5-bd73-da20b2b97327"). InnerVolumeSpecName "kube-api-access-hs4mh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:33:29.551484 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.551467 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56bac63b-37be-45e3-b6e8-034d0f2443ef-kube-api-access-c7kvz" (OuterVolumeSpecName: "kube-api-access-c7kvz") pod "56bac63b-37be-45e3-b6e8-034d0f2443ef" (UID: "56bac63b-37be-45e3-b6e8-034d0f2443ef"). InnerVolumeSpecName "kube-api-access-c7kvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 20:33:29.650910 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.650877 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1277a003-f9ae-4de5-bd73-da20b2b97327-plugins-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:29.650910 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.650903 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7kvz\" (UniqueName: \"kubernetes.io/projected/56bac63b-37be-45e3-b6e8-034d0f2443ef-kube-api-access-c7kvz\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:29.650910 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.650912 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hs4mh\" (UniqueName: \"kubernetes.io/projected/1277a003-f9ae-4de5-bd73-da20b2b97327-kube-api-access-hs4mh\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 28 20:33:29.928925 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.928890 2564 generic.go:356] "Generic (PLEG): container finished" podID="1277a003-f9ae-4de5-bd73-da20b2b97327" containerID="c6462d5ac46cfd38713d24ec4caa62c580965fc6705244909a048542ff2142e9" exitCode=0 Jul 28 20:33:29.929129 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.928936 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" Jul 28 20:33:29.929129 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.928985 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" event={"ID":"1277a003-f9ae-4de5-bd73-da20b2b97327","Type":"ContainerDied","Data":"c6462d5ac46cfd38713d24ec4caa62c580965fc6705244909a048542ff2142e9"} Jul 28 20:33:29.929129 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.929042 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k" event={"ID":"1277a003-f9ae-4de5-bd73-da20b2b97327","Type":"ContainerDied","Data":"0b5944254348ed5147b78c7883cff4804dab10b58883b8d407d95c337b90a452"} Jul 28 20:33:29.929129 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.929074 2564 scope.go:117] "RemoveContainer" containerID="c6462d5ac46cfd38713d24ec4caa62c580965fc6705244909a048542ff2142e9" Jul 28 20:33:29.930296 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.930275 2564 generic.go:356] "Generic (PLEG): container finished" podID="56bac63b-37be-45e3-b6e8-034d0f2443ef" containerID="ace87957632bd9ef71d1c4a060dbc087b2b4fd384b599dceaf8d5e6f63b5db84" exitCode=0 Jul 28 20:33:29.930378 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.930314 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" event={"ID":"56bac63b-37be-45e3-b6e8-034d0f2443ef","Type":"ContainerDied","Data":"ace87957632bd9ef71d1c4a060dbc087b2b4fd384b599dceaf8d5e6f63b5db84"} Jul 28 20:33:29.930378 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.930327 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" Jul 28 20:33:29.930378 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.930357 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds" event={"ID":"56bac63b-37be-45e3-b6e8-034d0f2443ef","Type":"ContainerDied","Data":"b620502d2215b996dfd1fe9c1c112562b5267f8d707408810c420836cef5f5ee"} Jul 28 20:33:29.940858 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.940840 2564 scope.go:117] "RemoveContainer" containerID="c6462d5ac46cfd38713d24ec4caa62c580965fc6705244909a048542ff2142e9" Jul 28 20:33:29.941294 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:33:29.941147 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6462d5ac46cfd38713d24ec4caa62c580965fc6705244909a048542ff2142e9\": container with ID starting with c6462d5ac46cfd38713d24ec4caa62c580965fc6705244909a048542ff2142e9 not found: ID does not exist" containerID="c6462d5ac46cfd38713d24ec4caa62c580965fc6705244909a048542ff2142e9" Jul 28 20:33:29.941294 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.941183 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6462d5ac46cfd38713d24ec4caa62c580965fc6705244909a048542ff2142e9"} err="failed to get container status \"c6462d5ac46cfd38713d24ec4caa62c580965fc6705244909a048542ff2142e9\": rpc error: code = NotFound desc = could not find container \"c6462d5ac46cfd38713d24ec4caa62c580965fc6705244909a048542ff2142e9\": container with ID starting with c6462d5ac46cfd38713d24ec4caa62c580965fc6705244909a048542ff2142e9 not found: ID does not exist" Jul 28 20:33:29.941294 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.941206 2564 scope.go:117] "RemoveContainer" containerID="ace87957632bd9ef71d1c4a060dbc087b2b4fd384b599dceaf8d5e6f63b5db84" Jul 28 20:33:29.949076 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.949000 2564 scope.go:117] "RemoveContainer" containerID="ace87957632bd9ef71d1c4a060dbc087b2b4fd384b599dceaf8d5e6f63b5db84" Jul 28 20:33:29.949271 ip-10-0-136-194 kubenswrapper[2564]: E0728 20:33:29.949250 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ace87957632bd9ef71d1c4a060dbc087b2b4fd384b599dceaf8d5e6f63b5db84\": container with ID starting with ace87957632bd9ef71d1c4a060dbc087b2b4fd384b599dceaf8d5e6f63b5db84 not found: ID does not exist" containerID="ace87957632bd9ef71d1c4a060dbc087b2b4fd384b599dceaf8d5e6f63b5db84" Jul 28 20:33:29.949355 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.949277 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ace87957632bd9ef71d1c4a060dbc087b2b4fd384b599dceaf8d5e6f63b5db84"} err="failed to get container status \"ace87957632bd9ef71d1c4a060dbc087b2b4fd384b599dceaf8d5e6f63b5db84\": rpc error: code = NotFound desc = could not find container \"ace87957632bd9ef71d1c4a060dbc087b2b4fd384b599dceaf8d5e6f63b5db84\": container with ID starting with ace87957632bd9ef71d1c4a060dbc087b2b4fd384b599dceaf8d5e6f63b5db84 not found: ID does not exist" Jul 28 20:33:29.959037 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.959013 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k"] Jul 28 20:33:29.961205 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.961183 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-47f87c-546f85554d-vg78k"] Jul 28 20:33:29.974307 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.974286 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds"] Jul 28 20:33:29.979393 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:29.979374 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-47f87c-67c54567fd-zvpds"] Jul 28 20:33:30.919123 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:30.919089 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1277a003-f9ae-4de5-bd73-da20b2b97327" path="/var/lib/kubelet/pods/1277a003-f9ae-4de5-bd73-da20b2b97327/volumes" Jul 28 20:33:30.919504 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:30.919440 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56bac63b-37be-45e3-b6e8-034d0f2443ef" path="/var/lib/kubelet/pods/56bac63b-37be-45e3-b6e8-034d0f2443ef/volumes" Jul 28 20:33:43.051906 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:43.051834 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-856746f77b-gl2jt_6023a46c-0039-4880-a083-7a3fe957b7c6/copy-manifests/0.log" Jul 28 20:33:43.058849 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:43.058823 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-856746f77b-gl2jt_6023a46c-0039-4880-a083-7a3fe957b7c6/manager/0.log" Jul 28 20:33:46.123473 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:46.123431 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xx8mx_0f26d5eb-a761-4b32-b0eb-31374b3c8cdc/manager/0.log" Jul 28 20:33:46.502277 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:46.502192 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp_c60d1426-7a09-40e7-b11d-7db3f87b0b05/istio-proxy/0.log" Jul 28 20:33:46.633109 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:46.633079 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ghhnq_1ca3d02d-68c6-44da-940e-1ccb46c31117/discovery/0.log" Jul 28 20:33:47.718986 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:47.718959 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-qlpp4_24e40f82-2efe-4646-b94a-5292a02636cf/main/0.log" Jul 28 20:33:47.824253 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:47.824229 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vwrhl_fd5dafea-e80c-4a87-b4ac-e6e8246ccab8/main/0.log" Jul 28 20:33:56.925920 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:56.925888 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2xb59_b5208c81-86ee-4635-9dd3-137393805762/global-pull-secret-syncer/0.log" Jul 28 20:33:57.152543 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:57.152512 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b7h24_5e23e056-ad6d-42d0-a5b2-54ebc60e3a9c/konnectivity-agent/0.log" Jul 28 20:33:57.216844 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:33:57.216753 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-194.ec2.internal_1eb1ec1d2f5d49622c78b5890bf891f2/haproxy/0.log" Jul 28 20:34:01.806128 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:01.806095 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xx8mx_0f26d5eb-a761-4b32-b0eb-31374b3c8cdc/manager/0.log" Jul 28 20:34:03.520870 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:03.520838 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_90214329-74d3-4151-bf44-dbbd73a33767/alertmanager/0.log" Jul 28 20:34:03.544463 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:03.544420 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_90214329-74d3-4151-bf44-dbbd73a33767/config-reloader/0.log" Jul 28 20:34:03.573448 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:03.573424 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_90214329-74d3-4151-bf44-dbbd73a33767/kube-rbac-proxy-web/0.log" Jul 28 20:34:03.598494 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:03.598470 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_90214329-74d3-4151-bf44-dbbd73a33767/kube-rbac-proxy/0.log" Jul 28 20:34:03.623837 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:03.623813 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_90214329-74d3-4151-bf44-dbbd73a33767/kube-rbac-proxy-metric/0.log" Jul 28 20:34:03.656179 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:03.656158 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_90214329-74d3-4151-bf44-dbbd73a33767/prom-label-proxy/0.log" Jul 28 20:34:03.683296 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:03.683234 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_90214329-74d3-4151-bf44-dbbd73a33767/init-config-reloader/0.log" Jul 28 20:34:03.768835 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:03.768808 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-tthgj_2cab676a-1e9b-408d-af0b-71d1dcca13f1/kube-state-metrics/0.log" Jul 28 20:34:03.794767 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:03.794743 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-tthgj_2cab676a-1e9b-408d-af0b-71d1dcca13f1/kube-rbac-proxy-main/0.log" Jul 28 20:34:03.821878 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:03.821855 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-tthgj_2cab676a-1e9b-408d-af0b-71d1dcca13f1/kube-rbac-proxy-self/0.log" Jul 28 20:34:03.864082 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:03.864059 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6d4fd8f47c-z8vjr_ba941b7e-7dd8-4162-b69a-6f7495855147/metrics-server/0.log" Jul 28 20:34:03.891550 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:03.891531 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-xxt9c_25774138-a7c7-43ae-a269-fd184cff3099/monitoring-plugin/0.log" Jul 28 20:34:04.064187 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.064160 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-99bqt_8e38cfa6-41d0-4f5c-b27f-1fb28751700a/node-exporter/0.log" Jul 28 20:34:04.100418 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.100395 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-99bqt_8e38cfa6-41d0-4f5c-b27f-1fb28751700a/kube-rbac-proxy/0.log" Jul 28 20:34:04.136122 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.136099 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-99bqt_8e38cfa6-41d0-4f5c-b27f-1fb28751700a/init-textfile/0.log" Jul 28 20:34:04.297661 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.297616 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-fcn9m_c73bd02b-752c-4648-8c0a-2c62a30980fb/kube-rbac-proxy-main/0.log" Jul 28 20:34:04.324384 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.324361 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-fcn9m_c73bd02b-752c-4648-8c0a-2c62a30980fb/kube-rbac-proxy-self/0.log" Jul 28 20:34:04.349011 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.348986 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-fcn9m_c73bd02b-752c-4648-8c0a-2c62a30980fb/openshift-state-metrics/0.log" Jul 28 20:34:04.388113 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.388090 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2c8b602a-4310-4a64-8b33-eb45a6929af4/prometheus/0.log" Jul 28 20:34:04.416565 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.416513 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2c8b602a-4310-4a64-8b33-eb45a6929af4/config-reloader/0.log" Jul 28 20:34:04.444475 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.444456 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2c8b602a-4310-4a64-8b33-eb45a6929af4/thanos-sidecar/0.log" Jul 28 20:34:04.470855 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.470813 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2c8b602a-4310-4a64-8b33-eb45a6929af4/kube-rbac-proxy-web/0.log" Jul 28 20:34:04.496274 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.496248 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2c8b602a-4310-4a64-8b33-eb45a6929af4/kube-rbac-proxy/0.log" Jul 28 20:34:04.520433 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.520411 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2c8b602a-4310-4a64-8b33-eb45a6929af4/kube-rbac-proxy-thanos/0.log" Jul 28 20:34:04.549830 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.549812 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2c8b602a-4310-4a64-8b33-eb45a6929af4/init-config-reloader/0.log" Jul 28 20:34:04.581590 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.581565 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-5f5wf_fd33efa7-a1e0-4e24-8d87-384198f9b658/prometheus-operator/0.log" Jul 28 20:34:04.604420 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.604400 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-5f5wf_fd33efa7-a1e0-4e24-8d87-384198f9b658/kube-rbac-proxy/0.log" Jul 28 20:34:04.635263 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.635237 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-k2r98_aaaa8139-2b99-4500-99a9-dc350205f405/prometheus-operator-admission-webhook/0.log" Jul 28 20:34:04.763454 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.763428 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68f88d7b94-cw5hp_91e557d8-c007-485b-96a2-a9f334099b1c/thanos-query/0.log" Jul 28 20:34:04.788497 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.788473 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68f88d7b94-cw5hp_91e557d8-c007-485b-96a2-a9f334099b1c/kube-rbac-proxy-web/0.log" Jul 28 20:34:04.814653 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.814577 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68f88d7b94-cw5hp_91e557d8-c007-485b-96a2-a9f334099b1c/kube-rbac-proxy/0.log" Jul 28 20:34:04.839021 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.838998 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68f88d7b94-cw5hp_91e557d8-c007-485b-96a2-a9f334099b1c/prom-label-proxy/0.log" Jul 28 20:34:04.864779 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.864756 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68f88d7b94-cw5hp_91e557d8-c007-485b-96a2-a9f334099b1c/kube-rbac-proxy-rules/0.log" Jul 28 20:34:04.890856 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:04.890837 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68f88d7b94-cw5hp_91e557d8-c007-485b-96a2-a9f334099b1c/kube-rbac-proxy-metrics/0.log" Jul 28 20:34:06.894754 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.894716 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v"] Jul 28 20:34:06.895182 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895108 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56bac63b-37be-45e3-b6e8-034d0f2443ef" containerName="payload-processing" Jul 28 20:34:06.895182 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895124 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="56bac63b-37be-45e3-b6e8-034d0f2443ef" containerName="payload-processing" Jul 28 20:34:06.895182 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895150 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" containerName="istio-proxy" Jul 28 20:34:06.895182 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895159 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" containerName="istio-proxy" Jul 28 20:34:06.895182 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895177 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4d795a9-d7ff-4db5-863c-e57ce8312c90" containerName="payload-processing" Jul 28 20:34:06.895182 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895186 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4d795a9-d7ff-4db5-863c-e57ce8312c90" containerName="payload-processing" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895207 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8" containerName="payload-pre-processing" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895215 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8" containerName="payload-pre-processing" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895226 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1277a003-f9ae-4de5-bd73-da20b2b97327" containerName="payload-pre-processing" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895234 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1277a003-f9ae-4de5-bd73-da20b2b97327" containerName="payload-pre-processing" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895243 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerName="istio-proxy" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895251 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerName="istio-proxy" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895264 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34e0c56e-8389-4d1d-8ec9-01fcc08d359a" containerName="istio-proxy" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895274 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="34e0c56e-8389-4d1d-8ec9-01fcc08d359a" containerName="istio-proxy" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895283 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7466d8d-0b76-48f9-bf86-931aa5ba0dd2" containerName="payload-processing" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895291 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7466d8d-0b76-48f9-bf86-931aa5ba0dd2" containerName="payload-processing" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895303 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4400cb29-8868-4e2f-9871-b82bf4e85dce" containerName="payload-pre-processing" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895311 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4400cb29-8868-4e2f-9871-b82bf4e85dce" containerName="payload-pre-processing" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895324 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34e0c56e-8389-4d1d-8ec9-01fcc08d359a" containerName="istio-proxy" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895332 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="34e0c56e-8389-4d1d-8ec9-01fcc08d359a" containerName="istio-proxy" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895344 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerName="istio-proxy" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895353 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerName="istio-proxy" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895363 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" containerName="istio-proxy" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895374 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" containerName="istio-proxy" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895462 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e37a3ae-49fa-49e6-87a5-4f4e11d8bde8" containerName="payload-pre-processing" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895475 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="34e0c56e-8389-4d1d-8ec9-01fcc08d359a" containerName="istio-proxy" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895483 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4400cb29-8868-4e2f-9871-b82bf4e85dce" containerName="payload-pre-processing" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895491 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerName="istio-proxy" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895499 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4d795a9-d7ff-4db5-863c-e57ce8312c90" containerName="payload-processing" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895509 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="56bac63b-37be-45e3-b6e8-034d0f2443ef" containerName="payload-processing" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895517 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7400e1b-ce61-4dc8-b0c0-9199f9c15395" containerName="istio-proxy" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895526 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="39d69163-a5aa-4dc2-ab6a-50e14e76e6c1" containerName="istio-proxy" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895533 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1277a003-f9ae-4de5-bd73-da20b2b97327" containerName="payload-pre-processing" Jul 28 20:34:06.895526 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895544 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="34e0c56e-8389-4d1d-8ec9-01fcc08d359a" containerName="istio-proxy" Jul 28 20:34:06.896969 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.895551 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7466d8d-0b76-48f9-bf86-931aa5ba0dd2" containerName="payload-processing" Jul 28 20:34:06.899115 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.899092 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:06.901867 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.901833 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0333bd86-bbee-4234-b331-e55c8ffd4e26-proc\") pod \"perf-node-gather-daemonset-f4f9v\" (UID: \"0333bd86-bbee-4234-b331-e55c8ffd4e26\") " pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:06.902009 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.901896 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0333bd86-bbee-4234-b331-e55c8ffd4e26-sys\") pod \"perf-node-gather-daemonset-f4f9v\" (UID: \"0333bd86-bbee-4234-b331-e55c8ffd4e26\") " pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:06.902009 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.901970 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0333bd86-bbee-4234-b331-e55c8ffd4e26-podres\") pod \"perf-node-gather-daemonset-f4f9v\" (UID: \"0333bd86-bbee-4234-b331-e55c8ffd4e26\") " pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:06.902009 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.901985 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bnmjx\"/\"kube-root-ca.crt\"" Jul 28 20:34:06.902009 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.902001 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7rkf\" (UniqueName: \"kubernetes.io/projected/0333bd86-bbee-4234-b331-e55c8ffd4e26-kube-api-access-p7rkf\") pod \"perf-node-gather-daemonset-f4f9v\" (UID: \"0333bd86-bbee-4234-b331-e55c8ffd4e26\") " pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:06.902221 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.902086 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bnmjx\"/\"default-dockercfg-8nlqz\"" Jul 28 20:34:06.902221 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.902097 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0333bd86-bbee-4234-b331-e55c8ffd4e26-lib-modules\") pod \"perf-node-gather-daemonset-f4f9v\" (UID: \"0333bd86-bbee-4234-b331-e55c8ffd4e26\") " pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:06.904017 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.903080 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bnmjx\"/\"openshift-service-ca.crt\"" Jul 28 20:34:06.908375 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:06.908351 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v"] Jul 28 20:34:07.002719 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:07.002684 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0333bd86-bbee-4234-b331-e55c8ffd4e26-podres\") pod \"perf-node-gather-daemonset-f4f9v\" (UID: \"0333bd86-bbee-4234-b331-e55c8ffd4e26\") " pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:07.002719 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:07.002721 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7rkf\" (UniqueName: \"kubernetes.io/projected/0333bd86-bbee-4234-b331-e55c8ffd4e26-kube-api-access-p7rkf\") pod \"perf-node-gather-daemonset-f4f9v\" (UID: \"0333bd86-bbee-4234-b331-e55c8ffd4e26\") " pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:07.002977 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:07.002773 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0333bd86-bbee-4234-b331-e55c8ffd4e26-lib-modules\") pod \"perf-node-gather-daemonset-f4f9v\" (UID: \"0333bd86-bbee-4234-b331-e55c8ffd4e26\") " pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:07.002977 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:07.002801 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0333bd86-bbee-4234-b331-e55c8ffd4e26-proc\") pod \"perf-node-gather-daemonset-f4f9v\" (UID: \"0333bd86-bbee-4234-b331-e55c8ffd4e26\") " pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:07.002977 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:07.002828 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0333bd86-bbee-4234-b331-e55c8ffd4e26-sys\") pod \"perf-node-gather-daemonset-f4f9v\" (UID: \"0333bd86-bbee-4234-b331-e55c8ffd4e26\") " pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:07.002977 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:07.002847 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0333bd86-bbee-4234-b331-e55c8ffd4e26-podres\") pod \"perf-node-gather-daemonset-f4f9v\" (UID: \"0333bd86-bbee-4234-b331-e55c8ffd4e26\") " pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:07.002977 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:07.002905 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0333bd86-bbee-4234-b331-e55c8ffd4e26-sys\") pod \"perf-node-gather-daemonset-f4f9v\" (UID: \"0333bd86-bbee-4234-b331-e55c8ffd4e26\") " pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:07.002977 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:07.002912 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0333bd86-bbee-4234-b331-e55c8ffd4e26-proc\") pod \"perf-node-gather-daemonset-f4f9v\" (UID: \"0333bd86-bbee-4234-b331-e55c8ffd4e26\") " pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:07.002977 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:07.002927 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0333bd86-bbee-4234-b331-e55c8ffd4e26-lib-modules\") pod \"perf-node-gather-daemonset-f4f9v\" (UID: \"0333bd86-bbee-4234-b331-e55c8ffd4e26\") " pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:07.012271 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:07.012250 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7rkf\" (UniqueName: \"kubernetes.io/projected/0333bd86-bbee-4234-b331-e55c8ffd4e26-kube-api-access-p7rkf\") pod \"perf-node-gather-daemonset-f4f9v\" (UID: \"0333bd86-bbee-4234-b331-e55c8ffd4e26\") " pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:07.212590 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:07.212486 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:07.351432 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:07.351371 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v"] Jul 28 20:34:07.355587 ip-10-0-136-194 kubenswrapper[2564]: W0728 20:34:07.355556 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0333bd86_bbee_4234_b331_e55c8ffd4e26.slice/crio-3bdd27ad101f8223885b2b9d8ec99ea6c264adb1aba338933e04339060506727 WatchSource:0}: Error finding container 3bdd27ad101f8223885b2b9d8ec99ea6c264adb1aba338933e04339060506727: Status 404 returned error can't find the container with id 3bdd27ad101f8223885b2b9d8ec99ea6c264adb1aba338933e04339060506727 Jul 28 20:34:07.441947 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:07.441917 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5fd674549c-qc52g_35fab412-7af6-45e9-a246-00946d6efe8f/console/0.log" Jul 28 20:34:08.117149 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:08.117116 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" event={"ID":"0333bd86-bbee-4234-b331-e55c8ffd4e26","Type":"ContainerStarted","Data":"3fd9f52abf7747389ed10f916fde773213061bf8e6daee9ba3bcfab0b439c0bc"} Jul 28 20:34:08.117562 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:08.117169 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" event={"ID":"0333bd86-bbee-4234-b331-e55c8ffd4e26","Type":"ContainerStarted","Data":"3bdd27ad101f8223885b2b9d8ec99ea6c264adb1aba338933e04339060506727"} Jul 28 20:34:08.117562 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:08.117228 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:08.146822 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:08.146770 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" podStartSLOduration=2.146754871 podStartE2EDuration="2.146754871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:34:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:34:08.145436548 +0000 UTC m=+3443.852648442" watchObservedRunningTime="2026-07-28 20:34:08.146754871 +0000 UTC m=+3443.853966763" Jul 28 20:34:09.338076 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:09.338042 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fd2mx_c17c0331-989d-43c0-9298-48a364bae22b/dns/0.log" Jul 28 20:34:09.361368 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:09.361340 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fd2mx_c17c0331-989d-43c0-9298-48a364bae22b/kube-rbac-proxy/0.log" Jul 28 20:34:09.551539 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:09.551513 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s2jhz_7e437f25-b6ba-42b8-b1c1-9245e8a86bc4/dns-node-resolver/0.log" Jul 28 20:34:10.149396 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:10.149365 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6kkrr_095953aa-1212-406f-b21e-7b649915699d/node-ca/0.log" Jul 28 20:34:11.141530 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:11.141496 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp_c60d1426-7a09-40e7-b11d-7db3f87b0b05/istio-proxy/0.log" Jul 28 20:34:11.290307 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:11.290271 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ghhnq_1ca3d02d-68c6-44da-940e-1ccb46c31117/discovery/0.log" Jul 28 20:34:12.416167 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:12.416136 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m9dst_accaa0ec-2c25-42e2-9065-392d7b2a0a23/serve-healthcheck-canary/0.log" Jul 28 20:34:13.223272 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:13.223240 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zhp49_e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1/kube-rbac-proxy/0.log" Jul 28 20:34:13.247248 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:13.247219 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zhp49_e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1/exporter/0.log" Jul 28 20:34:13.272801 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:13.272769 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zhp49_e3e25d20-f2d5-4d0d-8d93-5d98fc7fc0b1/extractor/0.log" Jul 28 20:34:14.132034 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:14.132008 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bnmjx/perf-node-gather-daemonset-f4f9v" Jul 28 20:34:16.620214 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:16.620183 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-856746f77b-gl2jt_6023a46c-0039-4880-a083-7a3fe957b7c6/manager/0.log" Jul 28 20:34:16.646832 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:16.646743 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-856746f77b-gl2jt_6023a46c-0039-4880-a083-7a3fe957b7c6/copy-manifests/0.log" Jul 28 20:34:17.485494 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:17.485462 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp_c60d1426-7a09-40e7-b11d-7db3f87b0b05/istio-proxy/0.log" Jul 28 20:34:17.640961 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:17.640929 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ghhnq_1ca3d02d-68c6-44da-940e-1ccb46c31117/discovery/0.log" Jul 28 20:34:18.106331 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:18.106303 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-cbcf9f5f9-rvbxp_25f26c6e-84cf-4e69-a8e6-97800ebc437f/manager/0.log" Jul 28 20:34:18.134794 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:18.134767 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-6slld_4d428e18-6930-4408-aa27-734824a79dbf/openshift-lws-operator/0.log" Jul 28 20:34:19.339924 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:19.339890 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2d5vp_c60d1426-7a09-40e7-b11d-7db3f87b0b05/istio-proxy/0.log" Jul 28 20:34:19.478799 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:19.478769 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ghhnq_1ca3d02d-68c6-44da-940e-1ccb46c31117/discovery/0.log" Jul 28 20:34:24.793664 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:24.793607 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gpbk6_9e333f58-159f-4604-aefa-55d3aeb458b6/kube-multus-additional-cni-plugins/0.log" Jul 28 20:34:24.818863 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:24.818834 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gpbk6_9e333f58-159f-4604-aefa-55d3aeb458b6/egress-router-binary-copy/0.log" Jul 28 20:34:24.843741 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:24.843714 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gpbk6_9e333f58-159f-4604-aefa-55d3aeb458b6/cni-plugins/0.log" Jul 28 20:34:24.869768 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:24.869742 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gpbk6_9e333f58-159f-4604-aefa-55d3aeb458b6/bond-cni-plugin/0.log" Jul 28 20:34:24.892444 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:24.892415 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gpbk6_9e333f58-159f-4604-aefa-55d3aeb458b6/routeoverride-cni/0.log" Jul 28 20:34:24.916447 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:24.916418 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gpbk6_9e333f58-159f-4604-aefa-55d3aeb458b6/whereabouts-cni-bincopy/0.log" Jul 28 20:34:24.940026 ip-10-0-136-194 kubenswrapper[2564]: I0728 20:34:24.939997 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gpbk6_9e333f58-159f-4604-aefa-55d3aeb458b6/whereabouts-cni/0.log"